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Managing Performance in the System of Support 

Sam Redding 

State education agencies (SEAs) find themselves in an unprecedented and pivotal position in leading change to 
improve student learning (Rhim & Redding, 2011). Governors and state legislators now view SEAs as engines for 
administering their high-priority school initiatives. The federal government now sets high expectations for state 
and local use of federal dollars while allowing greater state autonomy in achieving these goals. Local education 
agencies (LEAs), more readily than in the past, turn to their states for guidance and support in meeting their con- 
stituents’ demands for better school performance and in navigating the complexities of teaching and learning in an 
era of accountability. State leaders are attempting to shift their organizations to meet these demands, and innova- 
tive state leaders are finding new and bolder solutions to low performance, consistent with their state structures, 
traditions, and contexts (Redding, 2012). 

The state’s role has been evolving to meet this higher demand for several decades. The 1994 reauthorization of 
the Elementary and Secondary Education Act introduced the “statewide system of support” as a framework for 
guiding the state’s role in district and school oversight and assistance. This formative definition of the state’s work 
evolved into the more strategic and comprehensive System of Recognition, Accountability, and Support (SRAS), 
which identifies and coordinates key policy levers states have for assisting and intervening in low performing 
districts and schools. 

Managing the SRAS for performance will not be business as usual. States will need to be deliberate in designing 
and improving their SRAS to: 

1. Strategically mobilize all of its tools including building systemic and local capacity for change, creating 
opportunity for local agencies to pursue new strategies, offering incentives to improve, and intervening in 
the hardest cases; 

2. Differentiate their application of specific tools to deal with schools’ and districts’ varied performance 
challenges and trajectories; 

3. Include LEAs as both partners and targets of reform; 

4. Constantly assess its own efforts and identify ways to improve the SRAS to more effectively support 
improvement in student achievement outcomes. 

From Compliance to Performance Management in the SRAS 

SEAs are shifting from compliance-focused bureaucracies to more agile and proactive agencies that manage com- 
pliance in more productive ways while also catalyzing change. Many states began this transition with the state- 
initiated reform and standards movements of the 1980s and 1990s. The No Child Left Behind Act of 2002 moved 
all states onto this trajectory with its requirements for disaggregated student outcomes, performance targets, and 
public disclosure. The transition accelerated with the 2011 flexibility guidance from the Department of Educa- 
tion (DOE), and federal programs such as the re-vamped School Improvement Grants and Race to the Top. These 
efforts focused SEAs on high-leverage strategies, including: 

• college- and career-ready standards and assessments; 

• support for effective leadership and instruction, with performance-based staff evaluation; 

• greater differentiation in measures of progress and supports provided; 

• increased attention to high schools and graduation rates; and 

• turnaround principles for the lowest achieving schools. 

To various degrees, SEAs are migrating from funnels for funding streams and enforcers of regulation to catalysts 
for LEA and school improvement, although responsibility for assuring compliance persists (Rhim & Redding, 
2011). Increasingly, states are bringing coherence to disparate funding streams and programs, matching resources 
with operational need, and validating regulatory compliance with an eye towards effectiveness. They are moving 
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from random acts of technical assistance, scattered pro- 
grams and projects, and loose affiliations with external 
partners toward more intentional systems of recogni- 
tion, accountability, and support. They are examining 
student learning data and giving careful scrutiny of 
operational effectiveness, including assessing how the 
daily professional work of adults is impacting student 
outcomes. At the same time, the successful SEA is 
evaluating itself — including its system of recognition, 
accountability, and support — using the same rigorous 
performance metrics and evaluation tools that it expects 
of LEAs and schools. 

Strategically Mobilizing and Aligning the 
SRAS to Support Schools and Districts 

Most SEAs have re-purposed themselves as active cata- 
lysts for LEA and school improvement. Nonetheless, 
they continue to struggle with designing and managing 
an effective SRAS, responsive and flexible enough to 
deal with constantly shifting political environments and 
expectations and broad enough to incorporate the full 
scope of the state’s levers for change. 

An effective SRAS rests upon a coherent framework 
that includes five levers for change (Rhim, Elassel, & 
Redding, 2008): 

• Opportunities for improvement by reducing regu- 
latory burdens and encouraging innovation; 

• Incentives (positive and negative) for LEAs 
and schools to take the reins in their own 
improvement; 

• Systemic capacity development, including data 
and planning systems and policies that promote 
the supply of high quality leaders and teachers; 

• Local capacity development, diagnosing opera- 
tional effectiveness and professional practice and 
providing supports to address gaps 

• Interventions that direct the state’s most aggres- 
sive tools on turnaround towards the most persis- 
tently low achieving schools and districts. 

Table 1 provides examples of the five levers. 


Table 1: SEA Levers for Change in LEAs and 
Schools with Examples 

Lever for 
Change 

Examples 

Opportunity 

Routine scrutiny of state regula- 
tions to reduce burden; policies to 
encourage new starts (e.g., charter 
schools); policies to encourage 
innovation (e.g., charter-like dis- 
trict schools, pilot programs, light- 
house schools, innovative practice) 

Incentives 

Public disclosure of school perfor- 
mance; recognition of individual, 
district, and school achievements; 
discretionary grants; monetary 
rewards for individual, district, and 
school achievements 

Systemic 

Capacity 

Statewide data systems; web-based 
planning and implementation tools; 
pipelines for leaders and teach- 
ers; more effective credentialing 
requirements; alternate routes to 
certification 

Local Capacity 

Consultation, training, coaching, 
diagnostic site reviews 

Intervention 

Recovery districts; state takeover; 
staff replacement; turnaround 
models; external partners; school 
closure 


These levers work in concert to provide a rising tide 
that gently lifts most boats and more dramatically 
elevates others. The power of these levers is far greater 
than the sum of their individual parts. A strong SRAS 
exercises all five levers of change in a systematic and 
balanced manner, with careful attention to performance 
management by the SEA, the LEAs, and the schools. 

A key challenge in many states — stemming from weak 
authority, capacity or both — is the overreliance on one 
lever to the detriment of the others. For example, the 
ability of LEAs and schools to improve may be ham- 
pered by weak teacher and leader preparation programs 
in the universities or the state’s inattention to pipelines 
to supply LEAs and schools with leadership for turn- 
around. The state’s improvement planning process may 
be cumbersome and ineffective, and its data systems 
insufficient for the needs of LEAs and schools (e.g., 
cumbersome and not timely). 
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Further complicating matters, responsibility for the 
SRAS often spans multiple divisions across the SEA 
and relies on different statutory authorities and regula- 
tory powers. In some states, the SRAS may be pigeon- 
holed within the Title I office, with little integration 
with other offices within the SEA that impact school 
improvement. The SEA may view the SRAS as simply 
a technical assistance outreach to low-achieving 
schools and not consider the state policies and regula- 
tions that may be burdensome to LEAs and schools 
embarked upon focused improvement efforts. 

These factors make strategic alignment difficult but 
not impossible. As a starting point, SEAs need to 
conceptualize, position, and promote their SRAS as a 
real system that works to coherently support schools 
and districts, rather than a set of isolated parts moving 
independently. And like any system, an SRAS needs a 
manager — an individual or team who ensures its func- 
tioning and continued usefulness to the SEA’s strategic 
objectives related to student achievement. 

Differentiating the SRAS 

In differentiating its recognition, accountability, and 
support, the SRAS varies the opportunity, capacity- 
building supports, incentives, and interventions 
according to each LEA’s and school’s need. Typi- 
cally, high-capacity LEAs and schools respond well 
to incentives and greater opportunity (autonomy to 
innovate). While these same levers may be applied in 
varying degrees to stimulate improvement in low- 
performing LEAs and schools, low performance often 
signals the need for greater attention to local capacity 
development including talent, professional practice, 
and operational effectiveness. 

States are moving toward differentiated systems of 
support that vary the type, intensity, and duration of 
assistance along two metrics: 

1 . the LEA or school’s current level of per- 
formance as measured by student learning 
outcomes, especially on state standards assess- 
ments and in graduation rates, including disag- 
gregation by student groups; and 

2. the LEA’s or school’s particular operational 
strengths and weaknesses as determined by 
diagnosis of both disaggregated student learn- 
ing data and patterns of practice in the opera- 
tion of the LEA or school, particularly in areas 
of leadership, curriculum, and instruction. 


Student learning outcomes are used to classify all the 
schools in a state as: 

• consistently high achieving; 

• on a satisfactory trajectory of continuous 
improvement; 

• progressing at a significantly rapid pace; 

• in need of rapid improvement in order to 
reach a satisfactory trajectory of continuous 
improvement; 

• in need of turnaround — dramatic change to 
achieve significant improvement; or 

• candidate for closure or restart because of per- 
sistent low performance and lack of response to 
prior supports and interventions. 

For schools and LEAs on a satisfactory trajectory of 
continuous improvement, the state may provide an 
improvement process based on indicators of effective 
practice, self-assessed by LEA and school improve- 
ment teams. For schools and LEAs in need of rapid 
improvement, the state may introduce interventions, 
including those consistent with turnaround principles, 
alongside an improvement process based on indica- 
tors of effective practice. For schools in need of rapid 
improvement, self-assessment may be insufficient and 
may require more guidance in diagnosing current prac- 
tice and planning improvement. This guidance (coach- 
ing) in diagnosis and planning can be provided by the 
state, LEA, or external partner. 

Intervention 

SEA’s, with encouragement and funding from the 
USDOE, are targeting persistently low-achieving 
schools for interventions when milder improvement 
efforts have been unsuccessful. The intervention strate- 
gies include: (a) transformation (replacement of the 
principal, rigorous staff evaluation, and significant 
reforms); (b) turnaround (transformation strategies 
plus significant staff replacement); (c) re-start (charter 
schools and external management); and (d) closure 
(assignment of students to higher performing schools). 
The possibility of a state-initiated intervention can 
act as an incentive for LEAs and schools to conscien- 
tiously engage in substantial improvement, but it also 
provides a stopgap for the state for those schools that 
fail to improve despite concerted effort. 

State interventions differ in their reliance on turn- 
around partners, including the LEAs. In an LEA- 
managed turnaround, for example, the district is the 
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primary partner, working to replace staff and develop 
a turnaround strategy within the basic confines of local 
governance including collective bargaining agreements 
and district provided supports. In a state-managed 
turnaround, the state may bypass the LEA and take 
over management of the school directly, via a state- 
sponsored district or, more commonly, contracts with 
charter management organizations and operators to 
manage the school. 1 In a third variation, the SEA and 
LEA may assume shared responsibility for the design 
of the intervention. 

Recovery school districts place persistently low-achiev- 
ing schools in a state-managed district with intensive 
turnaround requirements, change in personnel, and sub- 
stantial scrutiny of operations. As states such as Loui- 
siana, Illinois, Tennessee, Indiana, Massachusetts, and 
Michigan gain experience with their recovery districts 
and similar interventions, more will be known about 
the effects on the LEAs from which the schools are 
removed, the ability of the state to build the capacity of 
the LEAs so that more schools don’t fall into this state 
of disrepair, the effective use of external providers like 
charter management organizations, and successful exit 
policies by which the state removes the school from the 
recovery district. 

One thing is for sure — change in personnel or gov- 
erning status will only be effective if they impact the 
proximal influences on student learning (Wang, Haer- 
tel, & Walberg, 1993). Especially, internal decision- 
making structures (leadership and teams) must be 
sound, rigorous, and nimble; instructional planning 
and delivery must comport with effective practice; 
student learning time must increase; teacher collabora- 
tive planning must increase in time and rigor; supports 
for student academic, social, and emotional learning 
must be strengthened; and family engagement must 
be improved. Intervention must result in dramatically 
reformed professional practice. 


1 A third approach, most clearly articulated in Colorado, 
focuses the state’s turnaround efforts at the district level 
and includes authority to reconstitute the LEA if per- 
formance challenges persist over a period of five years. 
While it is yet to be seen how this novel use of state 
oversight authority plays out, it has the potential to be 
more impactful by focusing more explicitly on district 
leadership — or lack thereof — as a lever for school 
improvement (something discussed more directly 
below). 


The Role of the Local Education Agency 
(LEA) 

LEAs are both the targets of the state’s system of 
recognition, accountability, and support and partners in 
implementation. Incentives, opportunity, capacity, and 
intervention are levers to be applied both to schools 
and LEAs. Just as a state may incentivize constructive 
change by rewarding success and providing conse- 
quences for failure, so can an LEA. States provide 
greater opportunity for change by removing regulatory 
barriers, granting greater local autonomy, and encour- 
aging innovation and “new space”; LEAs can do the 
same for their schools. States enhance the supply of 
human capital (leaders and teachers), and so can LEAs, 
especially in getting high-quality, motivated personnel 
in the schools that need them most. States provide rich 
and accessible data systems and planning processes, 
and LEAs can do likewise. Just as states differenti- 
ate supports to efficiently address diagnosed LEA and 
school operational deficits, LEAs can approach school 
improvement in the same manner. 

Effective state systems include the LEA as a central 
player in the improvement of its schools and give due 
attention to building the LEA’s capacity to do its part. 
This requires capacity at three levels (Lane, 2009): 

• the operational effectiveness of the central office 
and board in taking care of LEA functions; 

• the LEA’s infrastructure for school leadership, 
teaching, and learning; and 

• the LEA’s support for the improvement of indi- 
vidual schools. 

The state builds LEA capacity for improvement by 
providing supports at all three levels. Especially, the 
state ensures that the LEA applies its own differentiated 
supports for schools, including turnaround strategies 
and, in extreme cases, procedures for closure. When 
the state intervenes or provides support directly to a 
school, it includes the LEA as an integral participant in 
the activity, thus modeling an appropriate LEA role and 
building LEA capacity for school improvement. State 
policies and practices through its SRAS must specifi- 
cally provide LEAs with expectations and support rela- 
tive to their own operation and their leadership in the 
improvement of their schools. 

The advantage of SRAS focus on LEAs rather than 
schools is that state resources are more sufficient for 
the scope work, and as the LEA grows in its capac- 
ity to support school improvement, the state’s role 
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is lessened. Likewise, school improvement is more 
likely to be sustained if key elements of improvement 
are embedded in LEA policy and the LEA provides 
ongoing monitoring and support. One danger in 
an LEA-only focus is that resources, supports, and 
interventions may not be adequately targeted to the 
schools in greatest need of improvement or do not 
reach the school level with the necessary focus and 
power to effect change. Another pitfall in an LEA-only 
approach is that some LEAs are deficient in the capac- 
ity to do the work. A balanced state approach, tailored 
to the state’s context and engaging the LEA with its 
schools, seems the most appropriate. 

Assessing and Improving the SRAS 

The Building State Capacity and Productivity (BSCP) 
Center, with the appropriate Regional Comprehensive 
Center (RCC), provides technical assistance for SEAs 
to evaluate and improve the SRAS. The process is 
based on the rubric with 52 indicators in this docu- 
ment. The challenge for SEAs has been to achieve a 
high level of sustainable implementation often in a cli- 
mate of declining state resources and political change. 
The challenge for the BSCP Center and the RCCs is to 
help SEAs gain traction and achieve sustainable imple- 
mentation that produces results. 

The SRAS Performance Management Rubric is not a 
compliance monitoring process, a rating system, or a 
means of comparing one state’s system with another’s. 
Rather the state’s profile that is produced from using 
the performance management rubric contained in this 
SRAS document informs an SEA’s immediate plan- 
ning process by determining which indicators have 
priority status and are manageable. 

The authors of Managing Performance in the System 
of Support based this document on their previous 
work with statewide systems of support, including the 
Handbook on Statewide Systems of Support (Redding 
& Walberg, 2008), Coherence in the Statewide System 
of Support (Kerins, Perlman, & Redding, 2009), and 
other publications. The authors provided technical 
assistance to many states on their systems and piloted 
the rubric-based approach in nine states in the Acad- 
emy of Pacesetting States (2009-2011). Two of the 
authors served as reviewers for state applications for 
flexibility waivers, and another two assisted multiple 
states with their applications. The revised rubric -based 
document, published in 2013 by the BSCP Center, 
reflects lessons learned in this work as well as changes 
in federal guidance and state programs. 


Each SEA brings its own structure, tradition, and his- 
tory to the task of improving education. The SRAS 
Performance Management Rubric does not present a 
model for a system to support schools, but provides a 
framework within which many different strategies may 
fulfill the same purpose. 

The BSCP Center has adapted ADI’s Indistar® web- 
based planning and implementation tool for use by 
SEA teams with the SRAS Performance Management 
Rubric. This new system, called IndiSEA™, facilitates 
the important movement beyond initial assessment 
toward construction of a more effective SRAS. 

Conclusions 

A strong state system of recognition, accountability, 
and support will prune away ineffective programs, 
policies, and regulations as much as creating effective 
initiatives to spur LEA and school improvement. States 
adopting a systems approach to school improvement 
align their organizations, resources, and staff to fulfill 
their new performance-driven missions. These struc- 
tural alterations are a rudimentary form of performance 
management. Fine tuning the SRAS’s processes and 
system components in response to operational and out- 
come data is the next step. 

Striving for coherence and clarity of purpose is a prime 
factor in improving the state system, and achieving effi- 
ciencies is another. Designing, managing, and evaluat- 
ing the state’s system of recognition, accountability, 
and support enables the SEAs to not only do their work 
better, but also more efficiently. In a time of finite 
resources, performance and productivity are inherently 
intertwined. 
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Performance Management Rubric 

This Performance Management Rubric is revised from the version originally presented to the staff of nine states in 
the summer of 2009. Indicators have been added and the wording for other indicators has been revised in order to 
stay current with Department of Education policies. For each of these essential fifty-two indicators, please select 
the description in the cell that best describes your state’s profile in your present System of Recognition, Account- 
ability, and Support (SRAS). Note that in order to attain a score of “III,” the state SRAS must have met the condi- 
tions for getting a score of “II.” Similarly, in order to attain a score of “IV,” the SRAS has also met the conditions 
for attaining scores of “H” and “ITT.” 

The Priority, Opportunity, and Index in the first column enable SEA staff to declare the priority (how important 
is it to complete) as well as its opportunity (how easy is it to accomplish) for each indicator. Both ratings are on 
a “3” to “1” range. A “3” on opportunity means it is easier to accomplish since additional funds or legislative 
changes are not necessary. A “3” on priority means it is quite important for the SEA to work on this indicator. The 
Index Score is obtained by multiplying the opportunity and priority scores. The Index Score provides a way for 
SEA staff to sort these indicators for their planning in order to gain quick wins. More difficult items, and those of 
less priority, are still pursued , but the high-piority/high-opportunity items are given precedence. 

Part A: Design and Assessment of the SRAS 

1. SRAS design and differentiation 

Priority Score: 3 — highest priority, 2 — medium priority, 1 — lowest priority; Opportunity Score: 3 — rela- 
tively easy to address, 2 — accomplished within current policy and budget conditions, 1 — requires changes 
in current policy and budget conditions; Index Score: Priority Score x Opportunity Score 


Essential 

Indicators 

1 

II 

Ill 

IV 

No 

Development or 
Implementation 

Partial 

Development or 
Implementation 

Mostly 

Functional Level 
of Development 
and 

Implementation 

Full Level of 
Implementation 
and Evidence of 
Impact 

1.1 Designing 
and organizing 
an SEA System 
of Recognition, 
Accountability, and 
Support (SRAS) 

Priority 

Opportunity 

Index 

There is no 
identifiable, clearly 
defined system 
of recognition, 
accountability, and 
support for schools 
and districts. 

There is written, 
publicly available 
documentation, 
including the 
structure, goals, 
objectives, and 
timelines describing 
the SRAS and its 
available services 
and resources. 

The SEA has 
documentation 
(e.g., an operations 
manual), including 
an organization 
chart and description 
of roles and 
responsibilities 
for offices and 
personnel, both 
within and external 
to the SEA, that 
have responsibilities 
in implementing the 
SRAS. 

The SEA has an 
ongoing process 
for evaluating 
and improving 
the efficiency and 
effectiveness of its 
design, structure, 
and organization of 
the SRAS. 
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Essential 

Indicators 

1 

II 

Ill 

IV 

No 

Development or 
Implementation 

Partial 

Development or 
Implementation 

Mostly 

Functional Level 
of Development 
and 

Implementation 

Full Level of 
Implementation 
and Evidence of 
Impact 

1.2 Engaging 
stakeholders to 
solicit input on the 
development and 
improvement of the 
SRAS 

Priority 

Opportunity 

Index 

There is no 
evidence that the 
SEA provides an 
opportunity for 
stakeholders to 
comment about the 
development and 
improvement of the 
SRAS. 

There is written 
evidence 
documenting the 
initial input of 
key stakeholders 
in developing and 
improving the 
SRAS. 

The SEA has a 
documented process 
for stakeholder 
input in considering 
modifications to the 
SRAS. 

The SEA has 
a systematic 
process in place to 
obtain continuous 
feedback from key 
stakeholders in 
the development, 
improvement, 
and delivery of its 
SRAS. 

1.3 Managing the 
SRAS 

Priority 

Opportunity 

Index 

There is no 
documented 
description for 
the oversight and 
coordination of 
SRAS personnel, 
resources, and 
services. 

There is a 
documented 
description for 
the oversight and 
coordination of 
SRAS personnel, 
resources, and 
services. 

The SEA’s document 
describing its 
oversight and 
coordination of 
SRAS personnel, 
resources, and 
services is fully 
operational in 
practice. 

The SEA has an 
ongoing process 
for evaluating 
and improving 
the efficiency and 
effectiveness of its 
system for oversight 
and coordination of 
SRAS personnel, 
resources, and 
services. 

1.4 Staffing the 
SRAS 

Priority 

Opportunity 

Index 

The SEA has no 
documented policies 
and procedures for 
selecting, training, 
assigning, and 
evaluating personnel 
in the SRAS. 

The SEA has 
written policies 
and procedures for 
selecting, training, 
assigning, and 
evaluating personnel 
in the SRAS. 

The SEA has 
implemented 
its policies and 
procedures for 
selecting, training, 
assigning, and 
evaluating personnel 
in the SRAS. 

The SEA has an 
ongoing process 
for evaluating 
and improving 
the efficiency and 
effectiveness of 
its policies and 
procedures for 
selecting, training, 
assigning, and 
evaluating personnel 
in the SRAS. 
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Essential 

Indicators 

1 

II 

Ill 

IV 

No 

Development or 
Implementation 

Partial 

Development or 
Implementation 

Mostly 

Functional Level 
of Development 
and 

Implementation 

Full Level of 
Implementation 
and Evidence of 
Impact 

1.5 Integrating the 
SRAS within the 
SEA 

Priority 

Opportunity 

There is no evidence 
of a process by 
which the SEA 
integrates the 
functions of the 
SRAS into and with 
other offices and 
functions of the 
SEA. 

There is a 
documented 
description of the 
SEA’s ongoing 
efforts to integrate 
the functions of 
the SRAS into and 
with other offices 
and functions of the 
SEA. 

The SEA has 
integrated the 
functions of the 
SRAS into and with 
other offices and 
functions of the 
SEA. 

The SEA has an 
ongoing process 
for evaluating 
and improving 
the efficiency and 
effectiveness of 
its integration of 
functions of the 
SRAS into and with 
other offices and 
functions of the 
SEA. 

Index 

1.6 Differentiating 
support to districts 
and schools 

Priority 

Opportunity 

The SEA has no 
clearly defined 
procedures to 
determine which 
districts/schools 
receive SRAS 
services and 
resources and the 
amounts and types 
of resources and 
services provided. 

The SEA has 
developed clearly 
defined, written 
criteria (including 
student achievement, 
graduation rates, 
and professional 
practice*) and 
procedures to 
determine which 
districts/schools 
receive resources 
and services from 
the SRAS and the 
amounts and types 
of resources and 
services provided. 

The SEA has 
implemented clearly 
defined, written 
criteria (including 
student achievement, 
graduation rates, 
and professional 
practice) and 
procedures to 
determine which 
districts/schools 
receive resources 
and services from 
the SRAS and the 
amount and types 
of resources and 
services provided. 

The SEA has an 
ongoing process 
for evaluating 
and improving 
the efficiency and 
effectiveness of 
its criteria and 
procedures for 
differentiating the 
provision of SRAS 
resources and 
services to districts 
and schools. 

Index 


* Professional practice refers to the district or school’s practices of leadership, curriculum, assessment, 
instruction, family engagement, and similar operational practices relative to accepted standards. 
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Essential 

Indicators 

1 

II 

Ill 

IV 

No 

Development or 
Implementation 

Partial 

Development or 
Implementation 

Mostly 

Functional Level 
of Development 
and 

Implementation 

Full Level of 
Implementation 
and Evidence of 
Impact 

1.7 Improvement 
planning and 
implementation 
process for districts 
and schools 

Priority 

Opportunity 

Index 

The SEA has no 
clearly defined 
process for the 
districts/schools 
served by the 
SRAS to devise 
and implement 
plans to improve 
student achievement, 
graduation rates, 
and professional 
practice. 

The SEA has a 
clearly defined 
written process for 
districts and schools 
being served by its 
SRAS to develop 
and implement 
plans to improve 
student achievement, 
graduation rates, 
and professional 
practice. 

The SEA has 
implemented a 
system (perhaps 
electronic) to track 
local planning, 
implementation, and 
progress in meeting 
SEA performance 
targets and standards 
of professional 
practice so that 
districts/schools 
served by the SRAS 
receive appropriate 
support. 

The SEA has an 
ongoing process 
for evaluating 
and improving 
the efficiency and 
effectiveness of the 
district and school 
planning process 
and the SRAS’s 
monitoring and 
support of it. 

1.8 Providing 
differentiated 
services and 
resources to 
support district 
and school 
improvement 

Priority 

Opportunity 

Index 

There is no menu of 
available services 
and resources 
aligned to diagnosis 
of district and 
school performance, 
professional 
practice, and need. 

The SEA has a 
written menu of 
available services 
and resources 
aligned to diagnosis 
of district and 
school performance, 
professional 
practice, and need. 

The SEA’s 
differentiated and 
aligned services and 
resources to support 
district and school 
improvement are 
fully operational. 

The SEA has an 
ongoing process 
for evaluating 
and improving 
the efficiency and 
effectiveness of its 
differentiated and 
aligned services and 
resources to support 
district and school 
improvement. 

1.9 Intervening 
in districts and 
schools that 
repeatedly do 
not meet targets 
for student 
achievement and 
graduation rates 

Priority 

Opportunity 

Index 

There is no clear 
policy authorizing 
the SEA to directly 
intervene with 
districts/schools 
based on persistently 
unsatisfactory 
achievement and 
graduation rates. 

The SEA has 
written authority to 
intervene in districts/ 
schools because 
of persistently 
unsatisfactory 
achievement and 
low graduation 
rates, but lacks clear 
criteria to determine 
when or how to 
intervene. 

The SEA has a clear, 
documented policy 
and procedures 
for intervening in 
districts/schools 
with persistently 
unsatisfactory 
achievement and 
low graduation rates, 
including support for 
school turnarounds, 
and the policy and 
procedures are fully 
operational. 

The SEA has an 
ongoing process 
for evaluating 
and improving 
the efficiency and 
effectiveness of its 
intervention policies 
and procedures for 
districts/schools 
not demonstrating 
satisfactory 
achievement and / 
or graduation rates, 
including support for 
school turnarounds. 
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2. Supports and interventions for all students and subgroups 

Priority Score: 3 — highest priority, 2 — medium priority, 1 — lowest priority; Opportunity Score: 3 — rela- 
tively easy to address, 2 — accomplished within current policy and budget conditions, 1 — requires changes 
in current policy and budget conditions; Index Score: Priority Score x Opportunity Score 


Essential 

Indicators 

1 

II 

Ill 

IV 

No 

Development or 
Implementation 

Limited 
Development 
or Partial 
Implementation 

Mostly 

Functional Level 
of Development 
and 

Implementation 

Full Level of 
Implementation 
and Evidence of 
Impact 

2.1 Helping schools 
and districts better 
serve students with 
disabilities 

Priority 

Opportunity 

Index 

There are no formal 
SEA policies and 
procedures for 
assisting districts/ 
schools whose 
students with 
disabilities do not 
meet achievement 
targets. 

The SEA has 
developed written 
policies and 
procedures for 
assisting districts/ 
schools whose 
students with 
disabilities do not 
meet achievement 
targets. 

The SEA has 
implemented its 
written policies 
and procedures for 
assisting districts/ 
schools whose 
students with 
disabilities do not 
achievement targets. 

The SEA has an 
ongoing process 
for evaluating 
and improving 
the efficiency 
and effectiveness 
of policies and 
procedures for 
assisting districts/ 
schools whose 
students with 
disabilities do not 
meet achievement 
targets. 

2.2 Coordinating 
services for 
students with 
disabilities across 
SEA departments 
and programs 
to maximize 
service and reduce 
duplication 

Priority 

Opportunity 

Index 

There are no formal 
SEA policies and 
procedures for 
integrating its 
district/school 
assistance policies 
and programs to 
better serve students 
with disabilities. 

The SEA has 
developed written 
policies and 
procedures for 
integrating its 
district/school 
assistance programs, 
regardless of distinct 
funding sources, to 
better serve students 
with disabilities. 

The SEA has 
implemented its 
written plan for 
integrating its 
district/school 
assistance policies 
and programs, 
regardless of distinct 
funding sources, to 
better serve students 
with disabilities. 

The SEA has an 
ongoing process 
for evaluating 
and improving 
the efficiency 
and effectiveness 
of policies and 
procedures for 
integrating its 
district/school 
assistance programs, 
regardless of distinct 
funding sources, to 
better serve students 
with disabilities. 
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Essential 

Indicators 

1 

II 

Ill 

IV 

No 

Development or 
Implementation 

Limited 
Development 
or Partial 
Implementation 

Mostly 

Functional Level 
of Development 
and 

Implementation 

Full Level of 
Implementation 
and Evidence of 
Impact 

2.3 Helping schools 
and districts better 
serve English 
language learners 

Priority 

Opportunity 

Index 

There are no formal 
SEA policies and 
procedures for 
assisting districts/ 
schools whose 
English language 
learners fail to meet 
achievement targets. 

The SEA has 
developed written 
policies and 
procedures for 
assisting districts/ 
schools whose 
English language 
learners fail to meet 
achievement targets. 

The SEA has 
implemented its 
written policies 
and procedures for 
assisting districts/ 
schools whose 
English language 
learners fail to meet 
achievement targets. 

The SEA has an 
ongoing process 
for evaluating 
and improving 
the efficiency 
and effectiveness 
of policies and 
procedures for 
assisting districts/ 
schools whose 
English language 
learners fail to meet 
achievement targets. 

2.4 Coordinating 
services for English 
learners across 
SEA departments 
and programs 
to maximize 
service and reduce 
duplication 

Priority 

Opportunity 

Index 

There are no formal 
SEA policies and 
procedures for 
integrating its 
district/school 
assistance programs 
to better serve 
English learners. 

The SEA has 
developed written 
policies and 
procedures for 
integrating its 
district/school 
assistance programs, 
regardless of distinct 
funding sources, to 
better serve English 
learners. 

The SEA has 
implemented its 
written policies 
and procedures 
for integrating 
its district/school 
assistance programs, 
regardless of distinct 
funding sources, to 
better serve English 
learners. 

The SEA has an 
ongoing process 
for evaluating 
and improving 
the efficiency 
and effectiveness 
of policies and 
procedures for 
integrating its 
district/school 
assistance programs, 
regardless of distinct 
funding sources, to 
better serve English 
learners. 
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3. SRAS evaluation design 

Priority Score: 3 — highest priority, 2 — medium priority, 1 — lowest priority; Opportunity Score: 3 — rela- 
tively easy to address, 2 — accomplished within current policy and budget conditions, 1 — requires changes 
in current policy and budget conditions; Index Score: Priority Score x Opportunity Score 


Essential 

Indicators 

1 

II 

Ill 

IV 

No 

Development or 
Implementation 

Limited 
Development 
or Partial 
Implementation 

Mostly 

Functional Level 
of Development 
and 

Implementation 

Full Level of 
Implementation 
and Evidence of 
Impact 

3.1 Documenting 
district/school 
activities provided 
through SRAS 

Priority 

Opportunitv 

There are no written 
procedures to obtain 
documentation of 
how the SEA works 
with districts and 
schools through its 
SRAS. 

There are written 
procedures to collect 
documentation of 
SRAS work with 
schools and districts, 
(e.g., evidence 
of interventions, 
training, coaching), 
but they have not 
been implemented. 

The SEA has 
implemented its 
procedures to collect 
documentation of 
SRAS work with 
districts and schools. 

The SEA has an 
ongoing process 
for evaluating 
and improving 
the efficiency and 
effectiveness of 
procedures for 
documenting SRAS 
work with districts 
and schools. 

Index 


3.2 Evaluating the 
SRAS 

Priority 

Opportunitv 

There are no written 
evaluation policies 
and procedures 
to determine the 
effectiveness and 
efficiency of the 
SRAS. 

There are written 
evaluation policies 
and procedures, 
but they have not 
been implemented. 
Evaluation criteria 
include student 
achievement 
outcomes and 
district and school 
evaluations of SEA 
services. 

The SEA has 
implemented 
its policies and 
procedures for 
ongoing evaluation 
of the effectiveness 
and efficiency of its 
SRAS and releases 
periodic evaluation 
reports that are 
publicly available. 

The SEA has 
evidence that it has 
used the results 
of the evaluation 
process to improve 
its SRAS. 

Tndex 
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Essential 

Indicators 

1 

II 

Ill 

IV 

No 

Development or 
Implementation 

Limited 
Development 
or Partial 
Implementation 

Mostly 

Functional Level 
of Development 
and 

Implementation 

Full Level of 
Implementation 
and Evidence of 
Impact 

3.3 Evaluating the 
SEA’s assessment 
program 

Priority 

Opportunity 

There is no 
evidence that the 
SEA has policies 
and procedures 
to systematically 
evaluate its 
assessment program. 

The SEA has 
written policies 
and procedures 
to evaluate its 
assessment 
program, including 
alignment with 
the SEA’s content 
standards that would 
prepare students 
to take credit- 
bearing courses at 
post- secondary 
institutions or for a 
career and district/ 
school use of data 
in improvement 
planning. 

The SEA has 
implemented 
its policies and 
procedures for 
ongoing evaluation 
of its assessment 
program to ensure 
reliability and 
alignment with 
the SEA’s content 
standards and 
district/school use of 
data in improvement 
planning. 

The SEA 
systematically 
evaluates its 
assessment program 
to ensure the 
rigor, reliability, 
and validity of its 
test and that the 
test results can 
be meaningfully 
interpreted. Based 
on evaluations, 
it modifies 
assessments by 
using varying 
formats or 
modifying questions 
to improve rigor. 

Index 
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Part B: Resources and Services Provided to Districts and Schools 

4. District and school staff needs 

Priority Score: 3 — highest priority, 2 — medium priority, 1 — lowest priority; Opportunity Score: 3 — rela- 
tively easy to address, 2 — accomplished within current policy and budget conditions, 1 — requires changes 
in current policy and budget conditions; Index Score: Priority Score x Opportunity Score 


Essential 

Indicators 

1 

II 

Ill 

IV 

No 

Development or 
Implementation 

Limited 
Development 
or Partial 
Implementation 

Mostly 

Functional Level 
of Development 
and 

Implementation 

Full Level of 
Implementation 
and Evidence of 
Impact 

4.1 Enhancing the 
supply of teachers 
and leadership 
personnel 
skilled in school 
improvement 
strategies 

Priority 

Opportunity 

Index 

The SEA does not 
have formal policies 
and procedures to 
enhance the supply 
of teachers and 
leadership personnel 
skilled in school 
improvement 
strategies. 

The SEA has 
written policies 
and procedures 
for increasing the 
supply of teachers 
and leadership 
personnel skilled in 
school improvement 
strategies. 

The SEA has 
implemented its 
written policies and 
procedures using a 
variety of strategies 
(e.g., incentives, 
statutes, policies, 
and partnerships 
with institutions of 
higher education) 
to increase the 
supply of teachers 
and leadership 
personnel skilled in 
school improvement 
strategies. 

The SEA has an 
ongoing process 
for evaluating 
and improving 
the efficiency and 
effectiveness of 
its policies and 
procedures to 
increase the supply 
of teachers and 
leadership personnel 
skilled in school 
improvement 
strategies. 

4.2 Providing 
incentives and 
strategies for 
addressing a 
more equitable 
distribution of 
well-qualified and 
effective teachers 
within and across 
districts 

Priority 

Opportunity 

Index 

The SEA has 
no policies and 
procedures to 
address the equitable 
distribution of 
well-qualified and 
effective teachers 
within and across 
districts. 

The SEA has 
policies and 
procedures to 
encourage well- 
qualified and 
effective teachers 
to teach in schools 
identified as low 
achieving or having 
low-achieving 
subgroups and to 
ensure an equitable 
distribution of 
well-qualified and 
effective teachers 
within and across 
districts. 

The SEA has 
implemented 
its policies and 
procedures to place 
the well-qualified 
and effective 
teachers in schools 
identified as low 
achieving or having 
low-achieving 
subgroups and to 
ensure an equitable 
distribution of 
well-qualified and 
effective teachers 
within and across 
districts. 

The SEA has an 
ongoing process 
for evaluating 
and improving 
the efficiency 
and effectiveness 
of its policies 
and procedures 
to achieve 
more equitable 
distribution of 
qualified and 
effective teachers 
within and across 
districts. 
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Essential 

Indicators 

1 

II 

Ill 

IV 

No 

Development or 
Implementation 

Limited 
Development 
or Partial 
Implementation 

Mostly 

Functional Level 
of Development 
and 

Implementation 

Full Level of 
Implementation 
and Evidence of 
Impact 

4.3 Recruiting 
and retaining 
well-qualified and 
effective teachers 

Priority 

Opportunity 

Index 

The SEA has 
no policies and 
procedures to 
enhance recruitment 
and retention of 
well-qualified and 
effective teachers. 

The SEA has 
written policies 
and procedures to 
enhance recruitment 
and retention of 
well-qualified and 
effective teachers. 

The SEA has 
implemented 
its policies and 
procedures to 
enhance recruitment 
and retention of 
well-qualified and 
effective teachers. 

The SEA has an 
ongoing process 
for evaluating 
and improving 
the efficiency and 
effectiveness of 
its policies and 
procedures to 
enhance recruitment 
and retention of 
well-qualified and 
effective teachers. 

4.4 Recruiting and 
retaining effective 
district and 
school leadership 
personnel 

Priority 

Opportunity 

Index 

The SEA has 
no policies and 
procedures to 
enhance recruitment 
and retention of 
effective district and 
school leadership 
personnel. 

The SEA has 
written policies 
and procedures to 
enhance recruitment 
and retention of 
effective district and 
school leadership 
personnel. 

The SEA has 
implemented 
its policies and 
procedures to 
enhance recruitment 
and retention of 
effective district and 
school leadership 
personnel. 

The SEA has an 
ongoing process 
for evaluating 
and improving 
the efficiency and 
effectiveness of 
its policies and 
procedures to 
enhance recruitment 
and retention of 
effective district and 
school leadership 
personnel. 
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Essential 

Indicators 

1 

II 

Ill 

IV 

No 

Development or 
Implementation 

Limited 
Development 
or Partial 
Implementation 

Mostly 

Functional Level 
of Development 
and 

Implementation 

Full Level of 
Implementation 
and Evidence of 
Impact 

4.5 Engaging 
Institutions of 
Higher Education 
(IHEs) to better 
prepare new 
teachers and 
leadership 
personnel 

Priority 

Opportunity 

Index 

There are no 
written policies 
and procedures 
for the SEA and 
IHEs to jointly 
ensure that teacher 
and leadership 
programs prepare 
their students to 
understand relevant 
state policies, 
assessments, 
standards (e.g., 
the SEA’s college 
and career ready 
content standards), 
and effective 
professional 
practice. 

There are written 
policies and 
procedures for the 
SEA and IHEs to 
jointly ensure that 
future teachers and 
leadership personnel 
understand state 
standards, curricula, 
assessments, 
and effective 
professional 
practice. 

The SEA has 
implemented 
its policies and 
procedures for the 
SEA and IHEs to 
jointly ensure that 
future teachers and 
leadership personnel 
understand state 
standards, curricula, 
assessments, 
and effective 
professional 
practice. 

The SEA has an 
ongoing process 
for evaluating 
and improving 
the efficiency and 
effectiveness of 
its policies and 
procedures for the 
SEA and IHEs to 
jointly ensure that 
future teachers and 
leadership personnel 
understand state 
standards, curricula, 
assessments, 
and effective 
professional 
practice. The SEA 
collects information 
annually from newly 
placed teachers 
and administrators 
to evaluate if their 
collegiate experience 
has adequately 
provided them with 
the information 
to understand and 
implement SEA 
requirements. 

The summary 
information is 
shared with the 
IHEs. 
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Essential 

Indicators 

1 

II 

Ill 

IV 

No 

Development or 
Implementation 

Limited 
Development 
or Partial 
Implementation 

Mostly 

Functional Level 
of Development 
and 

Implementation 

Full Level of 
Implementation 
and Evidence of 
Impact 

4.6 Providing 
guidelines for 
the evaluation 
of teachers and 
principals 

Priority 

Opportunity 

Index 

There is no 
evidence that the 
SEA provides an 
opportunity for 
stakeholders to 
comment about the 
development of the 
SEA’s guidelines 
for teacher 
and leadership 
evaluations. 

As a result of 
stakeholder 
consultation, the 
SEA has developed 
and disseminated 
comprehensive and 
coherent information 
regarding teacher 
and leadership 
evaluation. 

The SEA holds 
awareness 
workshops to 
explain the 
evaluation 

guidelines (including 
consequences) 
and holds training 
programs to assist 
educators to use 
valid and reliable 
processes for staff 
evaluations. 

The SEA has an 
ongoing process 
for evaluating 
and improving 
the efficiency and 
effectiveness of 
its policies and 
procedures for 
assisting districts 
and schools with 
staff evaluation. 


5. Funding of improvement efforts 

Priority Score: 3 — highest priority, 2 — medium priority, 1 — lowest priority; Opportunity Score: 3 — rela- 
tively easy to address, 2 — accomplished within current policy and budget conditions, 1 — requires changes 
in current policy and budget conditions; Index Score: Priority Score x Opportunity Score 


Essential 

Indicators 

1 

II 

Ill 

IV 

No 

Development or 
Implementation 

Limited 
Development 
or Partial 
Implementation 

Mostly 

Functional Level 
of Development 
and 

Implementation 

Full Level of 
Implementation 
and Evidence of 
Impact 

5.1 Coordinating 
state and federal 
funding streams 
and programs 

Priority 

Opportunity 

Index 

There are no 
apparent policies 
and procedures 
to efficiently 
coordinate programs 
with different 
funding sources 
that are aimed at 
improving schools 
receiving SRAS 
services. 

The SEA has 
written policies 
and procedures to 
integrate multiple 
SEA and federal 
programs aimed at 
school improvement. 

The SEA has 
implemented 
its policies and 
procedures to 
integrate multiple 
programs with 
common goals but 
different funding 
streams in areas 
such as planning, 
resource allocation, 
training, reporting, 
and compliance 
monitoring. 

The SEA has an 
ongoing process 
for evaluating 
and improving 
the efficiency and 
effectiveness of 
its policies and 
procedures to 
integrate planning, 
resource allocation, 
training, reporting, 
and compliance 
monitoring across 
multiple programs 
with common goals 
but different funding 
streams. 
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Essential 

Indicators 

1 

II 

Ill 

IV 

No 

Development or 
Implementation 

Limited 
Development 
or Partial 
Implementation 

Mostly 

Functional Level 
of Development 
and 

Implementation 

Full Level of 
Implementation 
and Evidence of 
Impact 

5.2 Assisting 
districts in 
assessing their 
use of financial 
resources to fund 
improvement 
efforts 

Priority 

Opportunity 

Index 

The SEA has 
no process and 
procedures in place 
to help districts 
analyze their 
budgets to align 
financial resources 
with identified 
needs. 

The SEA has a 
documented process 
and procedures for 
facilitating local 
analysis of budgets, 
including written 
guidance on aligning 
financial resources 
with identified 
needs. 

The SEA provides 
budget advice, 
training, and 
support for districts 
to allocate their 
financial resources 
to improve student 
learning. 

The SEA has an 
ongoing process 
for evaluating 
and improving 
the efficiency and 
effectiveness of 
its policies and 
procedures of the 
SRAS services 
to help local staff 
evaluate, analyze, 
and reallocate 
resources to improve 
student learning. 
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6. Data analysis and use 

Priority Score: 3 — highest priority, 2 — medium priority, 1 — lowest priority; Opportunity Score: 3 — rela- 
tively easy to address, 2 — accomplished within current policy and budget conditions, 1 — requires changes 
in current policy and budget conditions; Index Score: Priority Score x Opportunity Score 


Essential 

Indicators 

1 

II 

Ill 

IV 

No 

Development or 
Implementation 

Limited 
Development 
or Partial 
Implementation 

Mostly 

Functional Level 
of Development 
and 

Implementation 

Full Level of 
Implementation 
and Evidence of 
Impact 

6.1 Providing a 
comprehensive 
SEA data system 

Priority 

Opportunity 

The SEA has a data 
system that meets 
minimum reporting 
requirements. 

The SEA is 
developing an 
integrated data 
system that reduces 
redundancy in 
data collection 
and that provides 
timely, accurate, 
and user-friendly 
data to inform 
school and district 
improvement. 

The SEA provides 
a timely, accurate, 
and integrated 
data system that 
reduces redundancy 
in data collection 
and which informs 
school and district 
improvement. The 
system provides 
a wide variety 
of indicators, 

(e.g., longitudinal 
trends for student 
subgroups, 
personnel, school 
report cards, 
attendance, 
graduation rates, 
and professional 
practice). 

The SEA has an 
ongoing process 
for evaluating 
and improving 
the efficiency and 
effectiveness of 
its data system 
(and the reduction 
of redundancy in 
data collection) 
that districts and 
schools use to 
inform improvement 
decisions. 

Index 

6.2 Using 
assessment data 

Priority 

Opportunity 

The SEA provides 
minimal guidance 
to districts and 
schools on use of 
assessment data in 
their improvement 
planning. 

The SEA has a 
clearly documented 
process for how 
district and school 
personnel can use 
both local and SEA 
assessment results 
for improvement. 

The SEA has 
implemented a 
training program to 
explain how district/ 
school staff can use 
assessment results to 
determine subgroup 
needs, provide 
differentiated 
services, and 
improve the 
educational 
program. 

The SEA has an 
ongoing process 
for evaluating 
and improving 
the efficiency and 
effectiveness of 
its policies and 
procedures for 
assisting districts 
and schools in using 
assessment data in 
their improvement 
planning. 

Index 
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7. Support teams and improvement consultants 

Priority Score: 3 — highest priority, 2 — medium priority, 1 — lowest priority; Opportunity Score: 3 — rela- 
tively easy to address, 2 — accomplished within current policy and budget conditions, 1 — requires changes 
in current policy and budget conditions; Index Score: Priority Score x Opportunity Score 


Essential 

Indicators 

1 

II 

Ill 

IV 

No 

Development or 
Implementation 

Limited 
Development 
or Partial 
Implementation 

Mostly 

Functional Level 
of Development 
and 

Implementation 

Full Level of 
Implementation 
and Evidence of 
Impact 

7.1 Matching 
districts/schools 
with support teams 
and district/school 
improvement 
consultants 

Priority 

Opportunity 

Index 

There are no 
systematic 
procedures for 
matching schools 
and districts with 
support teams and 
school improvement 
consultants based 
upon identified 
district/school 
needs and the 
qualifications of 
support teams and 
consultants. 

There are written 
policies and 
procedures for 
systematically 
matching districts/ 
schools with 
support teams 
and consultants 
based upon needs 
identified from 
student data and 
diagnosis of current 
professional 
practice and the 
qualifications of 
support teams and 
consultants. 

There is a systematic 
matching of schools 
and districts with 
support teams and 
consultants based 
on needs identified 
from student data 
and diagnosis of 
current professional 
practice and the 
qualifications of 
support teams and 
consultants. 

The SEA has an 
ongoing process 
for evaluating 
and improving 
the efficiency and 
effectiveness of 
its policies and 
procedures for 
matching support 
team and consultant 
qualifications with 
identified district 
and school needs, 
including evidence 
from district/school 
surveys and other 
data about the 
impact they have 
had in helping 
districts/schools to 
plan and implement 
strategies and 
structures for raising 
student achievement. 
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1 

II 

Ill 

IV 

Essential 

Indicators 

No 

Development or 
Implementation 

Limited 
Development 
or Partial 
Implementation 

Mostly 

Functional Level 
of Development 
and 

Implementation 

Full Level of 
Implementation 
and Evidence of 
Impact 

7.2 Training, 
supervising, 
and evaluating 
support teams 
and district/school 
improvement 
consultants 

Priority 

Opportunity 

There are no 
documented policies 
and procedures 
for training, 
supervising, and 
evaluating support 
teams and district/ 
school improvement 
consultants. 

The SEA has 
documented policies 
and procedures for 
training, supervising, 
and evaluating 
support teams and 
consultants. 

The SEA has 
implemented 
its policies and 
procedures for 
training, supervising, 
and evaluating 
support teams and 
consultants. 

The SEA has an 
ongoing process 
for evaluating 
and improving 
the efficiency 
and effectiveness 
of its policies 
and procedures 
for training, 
supervising, and 
evaluating support 
teams and district/ 
school improvement 
consultants. 

Index 





8. External partners and providers 

Priority Score: 3 — highest priority, 2 — medium priority, 1 — lowest priority; Opportunity Score: 3 — rela- 
tively easy to address, 2 — accomplished within current policy and budget conditions, 1 — requires changes 
in current policy and budget conditions; Index Score: Priority Score x Opportunity Score 


Essential 

Indicators 

1 

II 

Ill 

IV 

No 

Development or 
Implementation 

Limited 
Development 
or Partial 
Implementation 

Mostly 

Functional Level 
of Development 
and 

Implementation 

Full Level of 
Implementation 
and Evidence of 
Impact 

8.1 Managing 
and coordinating 
organizational 
partners 

Priority 

Opportunity 

Index 

There is no 
evidence that the 
SEA has formed 
partnerships with 
other organizations 
to further the goals 
of its SRAS. 

The SEA has 
written policies 
and procedures to 
create partnerships 
with entities 
outside the SEA, 
(e.g., universities, 
non-profit groups, 
businesses, civic 
organizations, 
and intermediate 
educational units). 

Multiple 

partnerships with 
entities outside the 
SEA have been 
implemented. These 
partnerships include 
clear guidance 
from the SEA 
regarding their role 
in school and district 
improvement. 

The SEA has an 
ongoing process 
for evaluating 
and improving 
the efficiency 
and effectiveness 
of its policies 
and procedures 
for creating 
and managing 
partnerships to assist 
districts and schools 
with improvement. 
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Essential 

Indicators 

1 

II 

Ill 

IV 

No 

Development or 
Implementation 

Limited 
Development 
or Partial 
Implementation 

Mostly 

Functional Level 
of Development 
and 

Implementation 

Full Level of 
Implementation 
and Evidence of 
Impact 

8.2 Providing 
guidance for 
tutoring and 
extended-learning 
time 

Priority 

Opportunity 

There are no policies 
and procedure to 
assist districts and 
schools in providing 
tutoring and 
extended-learning 
time for students. 

There are written 
policies and 
procedures for 
SRAS assistance to 
districts and schools 
in how they can 
provide tutoring and 
extended-learning 
time for students. 

The SEA policies 
and procedures for 
assisting districts 
and schools 
with tutoring 
and extended- 
learning time 
include materials 
and training 
opportunities for 
district and school 
staff and have been 
implemented. 

The SEA has an 
ongoing process 
for evaluating 
and improving 
the efficiency and 
effectiveness of 
its policies and 
procedures for 
assisting districts 
and schools in 
providing tutoring 
and extended- 
learning time for 
students. 

Index 


Part C: Implementation 

9. Removal of barriers to change and innovation 

Priority Score: 3 — highest priority, 2 — medium priority, 1 — lowest priority; Opportunity Score: 3 — rela- 
tively easy to address, 2 — accomplished within current policy and budget conditions, 1 — requires changes 
in current policy and budget conditions; Index Score: Priority Score x Opportunity Score 


Essential 

Indicators 

1 

II 

Ill 

IV 

No 

Development or 
Implementation 

Limited 
Development 
or Partial 
Implementation 

Mostly 

Functional Level 
of Development 
and 

Implementation 

Full Level of 
Implementation 
and Evidence of 
Impact 

9.1 Removing 
barriers to change 

Priority 

Opportunity 

The SEA has not 
developed policies 
and procedures 
to assist districts 
and schools in 
identifying and 
removing barriers to 
change. 

The SEA has formal, 
documented policies 
and procedures 
to assist schools 
and districts in 
identifying and 
removing barriers 
to change, (e.g., 
legislation, SEA 
board rules, 
facilitating 
alternate routes to 
certification, etc.). 

The SEA has 
implemented 
its policies and 
procedures to assist 
schools and districts 
in identifying 
and removing 
barriers to change, 
(e.g., legislation, 
SEA board 
rules, facilitating 
alternate routes to 
certification, etc.). 

The SEA has an 
ongoing process 
for evaluating 
and improving 
the efficiency and 
effectiveness of 
its policies and 
procedures for 
assisting districts 
and schools in 
identifying and 
removing barriers to 
change. 

Index 
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Essential 

Indicators 

1 

II 

Ill 

IV 

No 

Development or 
Implementation 

Limited 
Development 
or Partial 
Implementation 

Mostly 

Functional Level 
of Development 
and 

Implementation 

Full Level of 
Implementation 
and Evidence of 
Impact 

9.2 Creating 
options for new 
types of schools, 
including charter 
schools 

Priority 

Opportunity 

Index 

The SEA has not 
taken actions to 
create options for 
new types of schools 
via legislation and/ 
or development of 
written policies and 
procedures. 

Written policies and 
procedures have 
been developed for 
creating new types 
of schools (e.g., 
charter schools, pilot 
schools, lighthouse 
schools, schools 
within schools). 

The SEA has 
implemented 
policies and 
procedures for 
creating new types 
of schools (e.g., 
charter schools, pilot 
schools, lighthouse 
schools, schools 
within schools). 

The SEA has an 
ongoing process 
for evaluating 
and improving 
the efficiency and 
effectiveness of 
its policies and 
procedures for 
creating new types 
of schools in the 
state. 

9.3 Expanding 
access to college 
level courses or 
their prerequisites, 
dual enrollment 
courses, or other 
accelerated 
learning 
opportunities 

Priority 

Opportunity 

Index 

The SEA has not 
developed policies 
and procedures to 
work with districts, 
high schools, and 
IHEs to expand 
access to college 
level courses or 
prerequisites or 
other accelerated 
learning 

opportunities such as 
dual enrollment. 

The SEA has 
written policies and 
procedures to work 
with districts, high 
schools, and IHEs 
to encourage them 
to expand access to 
college level courses 
or prerequisites or 
other accelerated 
learning 

opportunities such as 
dual enrollment. 

The SEA has 
implemented 
its policies and 
procedures to work 
with districts, high 
schools, and IHEs 
to encourage them 
to expand access to 
college level courses 
or prerequisites or 
other accelerated 
learning 

opportunities such as 
dual enrollment. 

The SEA has an 
ongoing process 
for evaluating 
and improving 
the efficiency and 
effectiveness of 
its policies and 
procedures to work 
with districts, high 
schools, and IHEs 
to encourage them 
to expand access to 
college level courses 
or prerequisites or 
other accelerated 
learning 

opportunities such 
as dual enrollment. 
The SEA reports 
annually to the 
public on college- 
going and college 
credit-accumulation 
rates for all students 
and subgroups in 
each LEA and each 
public high school. 
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10. Incentives for change 

Priority Score: 3 — highest priority, 2 — medium priority, 1 — lowest priority; Opportunity Score: 3 — rela- 
tively easy to address, 2 — accomplished within current policy and budget conditions, 1 — requires changes 
in current policy and budget conditions; Index Score: Priority Score x Opportunity Score 


Essential 

Indicators 

1 

II 

Ill 

IV 

No 

Development or 
Implementation 

Limited 
Development 
or Partial 
Implementation 

Mostly 

Functional Level 
of Development 
and 

Implementation 

Full Level of 
Implementation 
and Evidence of 
Impact 

10.1 Setting 
consequences 
for low student 
achievement and 
low graduation 
rates 

Priority 

Opportunity 

Index 

The SEA has no 
consequences for 
low performing 
districts/schools. 

Written policies and 
procedures have 
been developed to 
levy consequences 
for low student 
achievement or 
graduation rates. 

Written policies and 
procedures have 
been implemented, 
and clear 

information has been 
provided to districts/ 
schools regarding 
consequences 
for low student 
achievement and 
graduation rates. 

The SEA has an 
ongoing process 
for evaluating 
and improving 
the efficiency and 
effectiveness of 
its policies and 
procedures relative 
to consequences 
for low student 
achievement and/or 
graduation rates. 

10.2 Providing 
positive incentives 
for improvement 

Priority 

Opportunity 

Index 

No positive 
incentives are 
provided districts 
or schools for 
improved academic 
achievement and 
graduation rates 
(e.g., special 
recognition, 
financial or other 
awards, and/or 
greater autonomy). 

The SEA has 
written policies 
and procedures 
for rewarding 
positive incentives 
to districts or 
schools for 
improved academic 
achievement and 
graduation rates, 
including the closing 
of achievement gaps 
for all subgroups of 
students. 

The SEA has 
implemented 
its policies and 
procedures for 
positive incentives 
to reward districts 
and schools with 
improved academic 
achievement and 
graduation rates, 
including the closing 
of achievement 
gaps. 

The SEA has an 
ongoing process 
for evaluating 
and improving 
the efficiency and 
effectiveness of 
its policies and 
procedures for 
positive incentives 
to reward districts 
and schools with 
improved academic 
achievement and 
graduation rates, 
including the closing 
of achievement 
gaps. 
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Essential 

Indicators 

1 

II 

Ill 

IV 

No 

Development or 
Implementation 

Limited 
Development 
or Partial 
Implementation 

Mostly 

Functional Level 
of Development 
and 

Implementation 

Full Level of 
Implementation 
and Evidence of 
Impact 

10.3 Publicly 
disclosing district 
and school 
performance 

Priority 

Opportunity 

Index 

The SEA annually 
discloses school and 
district performance 
data. 

Limited school and 
district data are sent 
to parents or are 
available at a public 
website. 

Data and reports 
are sent to parents, 
and the SEA’s 
website includes 
user-friendly and 
timely information. 
Communications to 
parents are made in 
multiple languages 
as appropriate. 

The SEA has an 
ongoing process 
for evaluating 
and improving 
the efficiency and 
effectiveness of 
its policies and 
procedures relative 
to public disclosure 
of district and school 
performance. 


11. SRAS communications 

Priority Score: 3 — highest priority, 2 — medium priority, 1 — lowest priority; Opportunity Score: 3 — rela- 
tively easy to address, 2 — accomplished within current policy and budget conditions, 1 — requires changes 
in current policy and budget conditions; Index Score: Priority Score x Opportunity Score 



1 

II 

Ill 

IV 

Essential 

Indicators 

No 

Development or 
Implementation 

Limited 
Development 
or Partial 
Implementation 

Mostly 

Functional Level 
of Development 
and 

Implementation 

Full Level of 
Implementation 
and Evidence of 
Impact 

11.1 

There are no 

There are written 

The SEA has 

The SEA has an 

Communicating 

documented policies 

policies and 

implemented its 

ongoing process 

with clear and 

and procedures 

procedures for 

written policies 

for evaluating 

systematic 

for a statewide 

communication 

and procedures for 

and improving 

communication 

communication 

among those who 

communication 

the efficiency and 

paths within the 

system among 

provide support such 

among those who 

effectiveness of 

SRAS 

those who provide 

as SEA employees, 

provide support such 

its policies and 

Priority 

Opportunity 

Tndex 

support such as SEA 
employees, regional 

regional offices, 
universities, and 

as SEA employees, 
regional offices, 

procedures relative 
to communication 

offices, universities, 
and other members 
of the SRAS. 

other members of 
the SRAS. 

universities, and 
other members of 
the SRAS. 

among those who 
provide support such 
as SEA employees, 
regional offices, 
universities, and 
other members of 
the SRAS. 
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1 

II 

Ill 

IV 

Essential 

Indicators 

No 

Development or 
Implementation 

Limited 
Development 
or Partial 
Implementation 

Mostly 

Functional Level 
of Development 
and 

Implementation 

Full Level of 
Implementation 
and Evidence of 
Impact 

11.2 Implementing 

There are no 

There are written 

The SEA has 

The SEA has an 

clear and 

documented policies 

policies and 

implemented its 

ongoing process 

systematic 

and procedures for 

procedures for 

written policies 

for evaluating 

communication 

communication 

communication 

and procedures for 

and improving 

paths between the 

among the SEA/ 

among the SEA/ 

communication 

the efficiency and 

SEA/SRAS and 

SRAS and districts/ 

SRAS and districts/ 

among the SEA/ 

effectiveness of 

districts/schools as 
well as significant 
others 

Priority 

Opportunity 

Index 

schools. 

schools. 

SRAS and districts/ 
schools. 

its policies and 
procedures relative 
to communication 
among the SEA/ 
SRAS and districts 
and schools. 


12. Technical assistance 

Priority Score: 3 — highest priority, 2 — medium priority, 1 — lowest priority; Opportunity Score: 3 — rela- 
tively easy to address, 2 — accomplished within current policy and budget conditions, 1 — requires changes 
in current policy and budget conditions; Index Score: Priority Score x Opportunity Score 



1 

II 

Ill 

IV 

Essential 

Indicators 

No 

Development or 
Implementation 

Limited 
Development 
or Partial 
Implementation 

Mostly 

Functional Level 
of Development 
and 

Implementation 

Full Level of 
Implementation 
and Evidence of 
Impact 

12.1 Delivering 

The SEA does 

The SEA has 

The SEA has 

The SEA has an 

training to 

not have formal, 

documented policies 

implemented 

ongoing process 

districts and 

documented policies 

and procedures 

its policies and 

for evaluating 

schools in school 

and procedures 

for training and 

procedures for 

and improving 

improvement 

for training and 

assisting districts/ 

training and 

the efficiency 

planning, 

assisting districts 

schools with 

assisting districts/ 

and effectiveness 

implementation, 

and schools in 

improvement 

schools with 

of its policies 

and monitoring 

improvement 

planning, 

improvement 

and procedures 


planning, 

implementation, and 

planning, 

for training and 

Priority 

Opportunity 

Index 

implementation, and 
monitoring. 

monitoring. 

implementation, and 
monitoring. 

assisting districts/ 
schools with 
improvement 
planning, 

implementation, and 
monitoring. 
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Essential 

Indicators 

1 

II 

Ill 

IV 

No 

Development or 
Implementation 

Limited 
Development 
or Partial 
Implementation 

Mostly 

Functional Level 
of Development 
and 

Implementation 

Full Level of 
Implementation 
and Evidence of 
Impact 

12.2 Providing 
technical assistance 
to improve 
professional 
practice 

Priority 

Opportunity 

The SEA does 
not have formal 
written policies and 
procedures for how 
it provides technical 
assistance to 
districts/schools to 
improve professional 
practice. 

The SEA has 
written technical 
assistance policies 
and procedures 
describing its 
technical assistance 
services relative to 
the improvement 
of professional 
practice, how 
the services are 
differentiated, and 
how districts and 
schools access them. 

The SEA has 
implemented its 
technical assistance 
policies and 
procedures for 
technical assistance 
services relative to 
the improvement 
of professional 
practice, how 
the services are 
differentiated, and 
how districts and 
schools access them. 

The SEA has an 
ongoing process 
for evaluating 
and improving 
the efficiency and 
effectiveness of 
its policies and 
procedures for 
providing technical 
assistance to districts 
and schools to 
improve professional 
practice. 

Index 

12.3 Building 
parent involvement 
into school 
improvement 

Priority 

Opportunity 

The SEA has no 
current written 
policies and 
procedures to 
provide guidance 
and training on 
how to include 
parents in the 
school improvement 
process. 

The SEA has current 
written policies 
and procedures to 
provide guidance 
and training 
opportunities 
for districts and 
schools on how to 
include parents in 
the improvement 
process. 

The SEA has 
implemented 
its policies and 
procedures to 
provide guidance 
and training 
opportunities 
for districts and 
schools on how to 
include parents in 
the improvement 
process. 

The SEA has an 
ongoing process 
for evaluating 
and improving 
the efficiency and 
effectiveness of 
its policies and 
procedures for 
providing guidance 
and training for 
districts and 
schools on how to 
include parents in 
the improvement 
process. 

Index 
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Essential 

Indicators 

1 

II 

Ill 

IV 

No 

Development or 
Implementation 

Limited 
Development 
or Partial 
Implementation 

Mostly 

Functional Level 
of Development 
and 

Implementation 

Full Level of 
Implementation 
and Evidence of 
Impact 

12.4 Evaluating 
external providers 

Priority 

Opportunity 

Index 

The SEA has 
no policies and 
procedures to 
conduct its own 
rigorous review of 
external providers 
used by the SEA and 
its LEAs to support 
district and school 
improvement and to 
assist districts and 
schools in doing the 
same. 

The SEA has 
policies and 
procedures to 
conduct its own 
rigorous review of 
external providers 
used by the SEA and 
its LEAs to support 
district and school 
improvement and to 
assist districts and 
schools in doing the 
same. 

The SEA has 
implemented 
its policies and 
procedures to 
conduct its own 
rigorous review of 
external providers 
used by the SEA and 
its LEAs to support 
district and school 
improvement and to 
assist districts and 
schools in doing the 
same. 

The SEA has an 
ongoing process 
for evaluating 
and improving 
the efficiency and 
effectiveness of 
its policies and 
procedures for 
conducting its own 
rigorous review of 
external providers 
used by the SEA and 
its LEAs to support 
district and school 
improvement and 
assisting districts 
and schools in doing 
the same. 

12.5 Implementing 
content standards 
that prepare 
students to take 
credit-bearing 
courses at post- 
secondary 
institutions and for 
a career 

Priority 

Opportunity 

Index 

The SEA has 
no policies and 
procedures for 
assisting districts 
and schools in 
implementing 
content standards 
that prepare students 
to take credit- 
bearing courses 
at post-secondary 
institutions and for a 
career. 

The SEA has 
written policies 
and procedures for 
assisting districts 
and schools in 
implementing 
content standards 
that prepare students 
to take credit- 
bearing courses 
at post-secondary 
institutions and for a 
career. 

The SEA has 
implemented 
its policies and 
procedures for 
assisting districts 
and schools in 
implementing 
content standards 
that prepare students 
to take credit- 
bearing courses 
at post-secondary 
institutions and for a 
career. 

The SEA has an 
ongoing process 
for evaluating 
and improving 
the efficiency and 
effectiveness of 
its policies and 
procedures for 
assisting districts 
and schools in 
implementing 
content standards 
that prepare students 
to take credit- 
bearing courses 
at post-secondary 
institutions and for a 
career. 
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13. Dissemination of knowledge 

Priority Score: 3 — highest priority, 2 — medium priority, 1 — lowest priority; Opportunity Score: 3 — rela- 
tively easy to address, 2 — accomplished within current policy and budget conditions, 1 — requires changes 
in current policy and budget conditions; Index Score: Priority Score x Opportunity Score 



1 

II 

Ill 

IV 

Essential 

Indicators 

No 

Development or 
Implementation 

Limited 
Development 
or Partial 
Implementation 

Mostly 

Functional Level 
of Development 
and 

Implementation 

Full Level of 
Implementation 
and Evidence of 
Impact 

13.1 Disseminating 

The SEA does not 

The SEA has 

The SEA has 

The SEA has an 

knowledge and/ 

have policies and 

policies and 

implemented 

ongoing process 

or research-based 

procedures for 

procedures for 

its policies and 

for evaluating 

practices 

making products and 

making products and 

procedures for 

and improving 


resources available 

resources available 

making products and 

the efficiency and 

Priority 

to help districts 

to help districts 

resources available 

effectiveness of 

and schools with 

and schools with 

to help districts 

its policies and 

Opportunity 

school improvement 

school improvement 

and schools with 

procedures for 

Index 

(e.g., manuals 

(e.g., manuals 

school improvement 

making products and 

on curriculum 

on curriculum 

(e.g., manuals 

resources available 


alignment, 

alignment, 

on curriculum 

to help districts 


instructional 

instructional 

alignment, 

and schools with 


improvement, 

improvement, 

instructional 

school improvement 


and parental 

and parental 

improvement, 

(e.g., manuals 


involvement). 

involvement). These 
products may be 
available from 
multiple sources. 

and parental 
involvement). 

on curriculum 

alignment, 

instructional 

improvement, 

and parental 

involvement). 

13.2 Producing 

The SEA does 

The SEA has 

The SEA has 

The SEA has an 

products and 

not produce and 

policies and 

implemented 

ongoing process 

resources to help 

disseminate products 

procedures for 

its policies and 

for evaluating 

districts and 

and resources to 

the production 

procedures for 

and improving 

schools improve 

help districts and 

and dissemination 

the production 

the efficiency and 


schools improve. 

of products and 

and dissemination 

effectiveness of 

Priority 


resources to help 
districts and schools 

of products and 
resources to help 

its policies and 
procedures for 

Opportunity 

Index 


improve. 

districts and schools 
improve and 
routinely produces 
and disseminates 
such products and 
resources. 

the production 
and dissemination 
of products and 
resources to help 
districts and schools 
improve. 
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14. Monitoring, program audits, and diagnostic site reviews 

Priority Score: 3 — highest priority, 2 — medium priority, 1 — lowest priority; Opportunity Score: 3 — rela- 
tively easy to address, 2 — accomplished within current policy and budget conditions, 1 — requires changes 
in current policy and budget conditions; Index Score: Priority Score x Opportunity Score 


Essential 

Indicators 

1 

II 

Ill 

IV 

No 

Development or 
Implementation 

Limited 
Development 
or Partial 
Implementation 

Mostly 

Functional Level 
of Development 
and 

Implementation 

Full Level of 
Implementation 
and Evidence of 
Impact 

14.1 Conducting 
state monitoring, 
program audits, 
and diagnostic site 
reviews 

Priority 

Opportunity 

The SEA has 
no policies and 
procedures for 
monitoring, 
conducting program 
audits, and providing 
diagnostic site 
reviews (including 
professional 
practice) in schools 
and districts 
identified as 
needing substantial 
improvement. 

The SEA has 
written policies 
and procedures 
for monitoring, 
conducting program 
audits, and providing 
diagnostic site 
reviews (including 
professional 
practice) in schools 
and districts 
identified as 
needing substantial 
improvement. 

The SEA has 
implemented 
its policies and 
procedures for 
monitoring, 
conducting program 
audits, and providing 
diagnostic site 
reviews (including 
professional 
practice) in schools 
and districts 
identified as 
needing substantial 
improvement. 

The SEA has an 
ongoing process 
for evaluating 
and improving 
the efficiency 
and effectiveness 
of its policies 
and procedures 
for monitoring, 
conducting program 
audits, and providing 
diagnostic site 
reviews (including 
professional 
practice) in schools 
and districts 
identified as 
needing substantial 
improvement. 

Index 

14.2 Documenting 
the status of 
districts/schools 

Priority 

Opportunity 

Districts/schools 
needing substantial 
improvement in 
student achievement 
and/or graduation 
rates have been 
identified, but it is 
not clear for how 
long or the rationale 
for placement. 

There is an 
annually updated 
identification of 
districts/schools 
needing substantial 
improvement, the 
number of years they 
have been identified, 
and an explanation 
of how they were 
identified, that is, 
the criteria the SEA 
used to identify 
these districts and 
schools. 

There is publicly 
available 
documentation 
explaining the 
criteria to remove 
districts and schools 
identified as low 
achieving or having 
low graduation 
rates, evidence 
documenting the 
number that have 
been removed, 
and analysis of the 
effectiveness of 
SRAS interventions 
and supports. 

The SEA has an 
ongoing process 
for evaluating 
and improving 
the efficiency 
and effectiveness 
of its policies 
and procedures 
for identifying 
districts/schools 
needing substantial 
improvement and 
reporting results of 
interventions and 
supports. 

Index 
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Essential 

Indicators 

1 

II 

Ill 

IV 

No 

Development or 
Implementation 

Limited 
Development 
or Partial 
Implementation 

Mostly 

Functional Level 
of Development 
and 

Implementation 

Full Level of 
Implementation 
and Evidence of 
Impact 

14.3 Monitoring 
the progress of 
individual districts/ 
schools 

Priority 

Opportunity 

The SEA has no 
process in place 
to monitor the 
progress of districts/ 
schools identified as 
needing substantial 
improvement in 
student achievement 
and/or graduation 
rates. 

The SEA has 
written policies and 
procedures for SEA 
staff and consultants 
to monitor identified 
districts/schools to 
ensure that they are 
implementing their 
improvement plans 
and receiving high- 
quality supports. 

The SEA has 
implemented its 
written policies and 
procedures for SEA 
staff and consultants 
to monitor identified 
districts/schools to 
ensure that they are 
implementing their 
improvement plans 
and receiving high- 
quality supports. 

The SEA has an 
ongoing process 
for evaluating 
and improving 
the efficiency and 
effectiveness of 
its policies and 
procedures for SEA 
staff and consultants 
to monitor identified 
districts/schools to 
ensure that they are 
implementing their 
improvement plans 
and receiving high- 
quality supports. 

Index 
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Part D: Outcomes for Districts and Schools Served by the SRAS 

15. Outcomes/results 

Priority Score: 3 — highest priority, 2 — medium priority, 1 — lowest priority; Opportunity Score: 3 — rela- 
tively easy to address, 2 — accomplished within current policy and budget conditions, 1 — requires changes 
in current policy and budget conditions; Index Score: Priority Score x Opportunity Score 


Essential 

Indicators 

1 

II 

Ill 

IV 

No 

Development or 
Implementation 

Limited 
Development 
or Partial 
Implementation 

Mostly 

Functional Level 
of Development 
and 

Implementation 

Full Level of 
Implementation 
and Evidence of 
Impact 

15.1 Establishing 

student 

achievement 

performance 

targets 

Priority 

Opportunity 

Index 

The SEA has 
not established 
performance targets 
for districts and 
schools, nor has it 
established criteria 
on how to use the 
state assessment to 
identify the highest 
performance and 
the high-progress 
schools. 

The SEA has 
established high 
performance 
targets for districts/ 
schools and criteria 
on how to use the 
state assessment to 
identify the highest 
performance and 
the high-progress 
schools, and targets 
and criteria include 
reliable and valid 
measures for 
determining student 
growth as well as the 
annual progress of 
schools and districts. 

The SEA has 
implemented its 
high performance 
targets for districts/ 
schools and criteria 
on how to use the 
state assessment to 
identify the highest 
performance and 
the high-progress 
schools, and targets 
and criteria include 
reliable and valid 
measures for 
determining student 
growth as well as the 
annual progress of 
schools and districts. 

The SEA has an 
ongoing process 
for evaluating 
and improving 
the efficiency 
and effectiveness 
of its policies 
and procedures 
relative to setting 
high performance 
targets and criteria 
for identifying the 
highest performance 
and the high- 
progress schools, 
and its means 
for reliable and 
valid measures for 
determining student 
growth as well as the 
annual progress of 
schools and districts. 

15.2 Addressing 
subgroup 
achievement gaps 

Priority 

Opportunity 

Index 

The SEA has a data 
system that can 
document subgroup 
achievement gaps 
over time but there 
is no systematic 
process for the SEA 
or districts to use 
that data. 

The SEA has a 
documented process 
and requirements 
for how schools 
and districts can 
use subgroup 
achievement gap 
data in district/ 
school improvement. 

The SEA has 
implemented its 
documented process 
and requirements 
for how schools 
and districts can 
use subgroup 
achievement gap 
data in district/ 
school improvement. 

The SEA has an 
ongoing process 
for evaluating 
and improving 
the efficiency and 
effectiveness of 
its policies and 
procedures for 
assisting districts 
and schools in 
closing achievement 
gaps. 
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Essential 

Indicators 

1 

II 

Ill 

IV 

No 

Development or 
Implementation 

Limited 
Development 
or Partial 
Implementation 

Mostly 

Functional Level 
of Development 
and 

Implementation 

Full Level of 
Implementation 
and Evidence of 
Impact 

15.3 Establishing 
student attendance 
performance 
targets 

Priority 

Opportunity 

Index 

The SEA has 
not established 
attendance 
performance targets 
for districts and 
schools. 

The SEA has 
established 
high attendance 
performance targets 
for districts/schools 
and has evidence 
to show that it can 
reliably and validly 
measure student 
attendance. 

The SEA has 
implemented its 
high attendance 
performance targets 
for districts/schools 
and provides 
evidence to show 
that it can reliably 
and validly measure 
student attendance. 

The SEA has an 
ongoing process 
for evaluating 
and improving 
the efficiency and 
effectiveness of 
its policies and 
procedures for 
setting attendance 
targets and 
providing evidence 
to show that it can 
reliably and validly 
measure student 
attendance. 

15.4 Establishing 
graduation rate 
performance 
targets 

Priority 

Opportunity 

Index 

The SEA has 
not established 
graduation 
performance targets 
for districts and 
schools. 

The SEA has 
established high 
graduation targets 
for districts/schools 
and has evidence 
to show that it can 
reliably and validly 
measure student 
graduation. 

The SEA has 
implemented high 
graduation targets 
for districts/schools, 
has evidence to 
show that it can 
reliably and validly 
measure student 
graduation, and 
provides a reliable 
and valid data 
system to document 
high school 
graduation rates and 
that information is 
made available to 
the public. 

The SEA has an 
ongoing process 
for evaluating 
and improving 
the efficiency 
and effectiveness 
of its policies 
and procedures 
for setting high 
graduation targets 
for districts/schools, 
showing evidence 
that it can reliably 
and validly measure 
student graduation, 
and providing a 
reliable and valid 
data system to 
document high 
school graduation 
rates and make 
this information 
available to the 
public. 
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Assessing and Improving the SEA SRAS 
Planning Based on the Assessment 

The completion of the Performance Management Rubric is a necessary first step for SEA staff as they organize 
their efforts to improve their System of Recognition, Accountability, and Support for districts and schools. How- 
ever, the next step is to take these data and move toward a plan for action. The Academic Development Institute 
has created an online tool (Indistar®) which provides a convenient way for the SEA to monitor progress on the 
Performance Management Rubric. This combination of tool plus rubric developed in consultation with the Build- 
ing State Capacity and Productivity Center is IndiSEA™. The following paragraphs provide a brief overview of 
how the main components of IndiSEA™ is meant to be used. The online tool also includes other resources such as 
live reports and a Meeting/Agenda set up feature. 

SRAS Online Tool Overview of Assessing, Planning, and Monitoring: 

Assess — Team Assesses Indicators 

Assessing the indicators is the initial step of the process. IndiSEA™ lists 52 indicators, along with the Perfor- 
mance Management Rubric information. The SEA team decides on their implementation status and the priority 
and opportunity for each indicator. The team uses the Wise Ways®/Exemplars and the rubric to rate their current 
level of implementation (No Development, Limited Development, or Full Implementation) on a scale of 1 to 4. 
Since a rubric score of 2 and 3 are in the middle of the scale, they both fall in the ‘Limited Development’ category 
in the online tool. SEA teams use the Performance Management Rubric and the Wise Ways®/Exemplars to guide 
discussion, and a process manager enters information regarding implementation and evidence into IndiSEA™. 

Create — Team Creates and Revises Objective Plans 

Once the team has assessed the indicators, IndiSEA™ ranks the indicators (now called objectives) by the Index 
score. The Index score is the product of the priority score and the opportunity score. The SEA team decides on the 
order to plan and implement their objectives. For each objective the team works on, they assign a team member 
to manage the work, they create a statement of what the objective will look like at lull implementation, and set a 
target date. The team then creates tasks that will help them reach full implementation. 

Monitor — Team Monitors Progress 

Once objectives have tasks, the SEA team implements and tracks the progress of their SEA level plan. As tasks 
are completed, the team discusses evidence of lull implementation to determine if they have reached their objec- 
tive. Indicators and objectives can be continually planned and monitored, as this is meant to be a continuous 
improvement model. 
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Explaining the Rubric 

Susan Hanes, C. Thomas Kerins, Carole Perlman, & Steven Ross 


Introduction 

This section explains each Essential Indicator more fully by using examples of activities the authors found 
when visiting Alabama, Kentucky, Ohio, Tennessee, and Washington. Additional studies of Montana, Virginia, 
Idaho, and Oklahoma were also completed, so these reports are referenced as well. Additional information from 
a national survey conducted by the authors is also included for illustrative purposes. For example, Indicator 
4.2 states that an SEA provides: “incentives and strategies for addressing a more equitable distribution of well 
qualified and effective teachers within and across districts.” A detailed example derived from interviews in Ohio 
explains what is meant by this indicator. For each indicator, the state that is the source of the information has 
been identified. Some of these states’ practices may have changed or expanded since the original information was 
provided. Also included are examples from 2012 flexibility waiver requests that were submitted to the USDE for 
approval. 

The second section provides illustrations to help users understand the issues involved in deciding how to score 
their SEA’s placement on the rubric’s scale. Of particular importance are the issues involved in differentiating 
between Categories II and III as well as III and IV. The SEA responses to the survey of all state systems of support 
provide the basis for the exemplars. However, for many of the indicators, the responses did not provide sufficient 
information to create a meaningful example. In these cases, elaborations create richer illustrations. For that reason, 
individual SEAs are not always referenced. 

By supplying this level of detail, users will be able to make reliable and valid judgments of the status of their SEA 
System of Recognition, Accountability, and Support (SRAS). The rubric can be used over time to document posi- 
tive changes by individual indicator and section as well as document of changes throughout the SRAS. 

Illustrating the Indicators 

Part A: Design and Assessment of the SRAS 

1. SRAS design and differentiation 

1.1 Designing and organizing an SEA System of Recognition, Accountability, and Support (SRAS) 

The performance levels for this indicator range from having no identifiable, clearly defined system of support 
for schools and districts with academic problems to the SEA having role descriptions for each person and office 
within the SRAS as well as a process for assessing and improving the efficiency and effectiveness of its own 
structure. 

Alabama faced the daunting task of trying to unify and coordinate the state’s support services. The goal was to 
ensure that all Alabama schools needing improvement received coordinated assistance from a unified delivery 
system. 

To accomplish this, the Deputy Superintendent charged the directors of several departments to work together. 

She then organized an Accountability Roundtable whose mission was to provide a coordinated, seamless system 
of continual technical assistance and support to schools in the areas of curriculum, instruction, fiscal responsibil- 
ity, management, and leadership. Technical assistance to schools as defined by the state accountability legislation 
is coordinated through the Accountability Roundtable and provided by the State Support Team (SST). 1 The SST 
provides technical assistance and support to all districts/schools with focused assistance to districts/schools that do 
not make performance targets. 


1 The State Support Team (SST) is comprised of the SEA staff of the Instructional Services Division as well as the Alabama Math and Sci- 
ence Technology Initiative (AMSTI) Site Directors and Math and Science Specialists, Alabama Reading Initiative (ARI) Regional Partners 
and Regional Reading Coaches, Regional School Improvement Coaches, and Peer Mentors. 
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1 .2 Engaging stakeholders to solicit input on the 
development and improvement of the SRAS 

The performance levels for this indicator range from 
no evidence of opportunities for stakeholders to review 
and comment about the development and improvement 
of the SRAS to the SEA having a systematic pro- 
cess in place to obtain continuous feedback from key 
stakeholders. 

In Kentucky, the process of improvement planning 
focuses school and district improvement efforts on 
student needs by bringing together all stakeholders to 
plan for improvement, by focusing planning efforts on 
priority needs and closing achievement gaps between 
subgroups of students, by building upon school and dis- 
trict capacity for high quality planning, and by making 
connections between the funds that flow into districts. 

1.3 Managing the SRAS 

The performance levels for this indicator range from 
having no clearly identifiable plan for oversight or 
coordination to a systematic process for coordinating 
planning and resource allocation, including evidence of 
effects. 

Kentucky has implemented integrated planning among 
programs to coordinate services including their system 
of ETighly Skilled Educators (ElSEs). They organize 
identified schools around a common focus of improved 
student learning using the Kentucky Scholastic Audit 
and Review process — a comprehensive analysis con- 
ducted by the HSEs — of the learning environment, effi- 
ciency, leadership, culture, and academic performance 
of schools and districts. 

In terms of organizing people, Kentucky has the 
Instructional Support Network to build the capacity of 
district administrators to provide leadership in making 
connections between planning for instruction and plan- 
ning for professional development; it is these connec- 
tions that provide the foundation for continuous school 
improvement. One example of coordination is the 
Voluntary Partnership Assistance Teams. These teams 
focus on the districts most in need of assistance. SEA 
staff, along with an E1SE, join a school board member, 
selected by the Kentucky School Board Association, 
and an administrator, chosen by the Kentucky Associa- 
tion of School Administrators, to make up a team. All 
the members of each team are from districts that have 
succeeded in raising student achievement. The super- 
intendent of the struggling district is the chair of the 
group. 


Finally, Kentucky has a Bureau Leadership Planning 
Team composed of the commissioner, the deputy, and 
all associates. They meet at least monthly to discuss, 
review, and evaluate the SRAS. Additionally, two key 
associate commissioners share the responsibility for 
day-to-day activities around statewide support. The 
SEA relies heavily on its approach of cross-agency 
decision making, maximizing limited resources, and 
shared ownership of initiatives and responsibilities. 

1.4 Staffing the SRAS 

The performance levels for this indicator range from a 
lack of documented policies for selecting, training, and 
assigning personnel into the SRAS to the SEA having 
an ongoing process for the selection, training, assign- 
ment, and evaluation of SRAS staff. 

Due to the lack of local capacity, the use of coaches 
is a central feature of Montana’s improvement pro- 
cess. The coach position is designed to facilitate the 
changes at the local level by empowering community 
members to lead the changes. The Montana manage- 
ment team called upon a number of retired principals, 
administrators, school board members, and consul- 
tants to fill some of the positions. Other coaches, who 
were employed by districts at the time, took a leave 
of absence from their current positions to temporarily 
serve as coach in one of Montana’s Schools of Promise. 

These on-site field staff positions require knowledge of 
effective schools research in teaching and learning and 
successful instructor designs, classroom implementa- 
tion and school administration, along with the ability to 
deliver appropriate K-12 education to diverse students. 
Montana leaders recruited and placed on-site field staff 
not only based on their skills, but also on the ability to 
fit within the local communities (Corbett, 2011). 

1.5 Integrating the SRAS within the SEA 

The performance levels for this indicator range from 
no evidence of a process by which the SEA integrates 
the functions of the SRAS into other SEA offices to 
the SEA having an ongoing process for evaluating and 
improving the efficiency and effectiveness of its inte- 
gration of functions with other SEA offices. 

Colorado uses a cross-functional team of SEA staff to 
forge three-year partnerships with a number of their 
districts. In Wyoming, special education personnel are 
part of the support system. They align Rtl and techni- 
cal assistance delivery with the results from special 
education monitoring. Special education personnel are 
members of Wyoming’s technical assistance teams and 
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consult with School Improvement personnel. In Ari- 
zona, English learner facilitators participate in all LEA 
Resource Team district visitations. 

Idaho developed and implemented a coordinated 
SRAS that involves IHEs as intermediate agencies 
and consists of multiple support strategies, network- 
ing opportunities, a revamped planning process, and 
specific processes to identify needs and target services 
in high need sites. Sixty-one schools and 30 districts 
are formally part of the Idaho Building Capacity 
project, which serves as the hub of support for targeted 
districts and schools. Additionally, 467 schools and 
76 districts are accessing key features of this SRAS, 
including the Wise Tool (ADI’s web-based school 
improvement process) and networking opportunities 
such as the Principals’ Leadership Academy and the 
Superintendents’ Network. This was made possible by 
coordinating Title I and Special Education programs in 
the same department (Lane, 2010). 

1.6 Differentiating support to districts and schools 
The performance levels for this indicator range from 
the state having no clearly defined procedures to 
determining which districts/schools receive SRAS 
services and resources to the state having evidence of 
an ongoing process illustrating that it provides differ- 
ent levels and amounts of intensive support based on 
student performance. This process is constantly being 
evaluated to improve the efficiency and effectiveness 
of its procedures. 

Tennessee implemented a sophisticated approach 
to providing differentiated services. First, the state 
launched a program of Exemplary Educators (EE) who 
serve as mentors to principals and teachers, analyze 
student performance data, connect with professional 
development providers, and build capacity for continu- 
ous school improvement. A major part of their training 
prepares them to provide different degrees and levels 
of support according to the needs of the districts/ 
schools. 

Secondly, Tennessee relies on its publication, What 
is a Good School Appraisal Guide and Rubric, as the 
focus of its work with districts on curriculum, instruc- 
tion, organization, and assessments (both formative 
and summative). The Guide is used as the basis of 
site visits which result in an individual school pro- 
file of strengths and areas of planning needs. The EE 
personnel use these profiles to guide their assistance. 
The Appraisal Guide employs a set of criteria defin- 
ing effective schools as well as a connected set of 


standards and measurement statements with matched 
rubrics. 

Districts identified as not meeting the required 
accountability benchmarks must annually address cer- 
tain components in their consolidated application for 
federal funds. To help local personnel do this, EEs are 
assigned to these districts and schools based on their 
specific needs. Training enables the EEs to provide dif- 
ferentiated services. EEs help identified districts orga- 
nize their priorities and respond to a series of questions 
about important issues such as the gap between their 
present curricular practices and what they believe they 
should do to increase student learning. This analysis 
includes reflective questions for staff on how they 
can better use their time, money, personnel, and other 
resources to make necessary changes. The subsequent 
analysis of the data provides an opportunity to pinpoint 
where technical assistance and differentiated services 
are needed. 

The Tennessee process requires these local person- 
nel to collect, disaggregate, analyze, and synthesize 
the data to focus on improving student achievement. 
Subsequently, every local goal must be linked to data. 
Tennessee provides EE services to assist these local 
educators until the school is off the high priority list. 

In its 2012 flexibility waiver request, Florida noted 
that it uses technical assistance from regional Dif- 
ferentiated Accountability instructional coaching 
staff to help districts/schools in the areas of reading, 
mathematics, science, data, Rtl, career and technical 
education, STEM (science, technology, engineering, 
and mathematics), Florida Continuous Improvement 
model, effective instruction, content area literacy, and 
effective coaching (Florida, 2012). 

1.7 Improvement planning and implementation 
process for districts and schools 
The performance levels for this indicator range from 
no clearly defined process for districts/schools served 
by the SRAS for devising and implementing plans to 
improve student achievement, graduation rates, and 
professional practice to the state showing evidence 
that schools and districts served by its SRAS have an 
effective planning process, and the SEA continuously 
monitors it. 

The state of Washington uses the School Improvement 
Planning Process Guide to lead principals and school 
communities through a cycle of continuous improve- 
ment. The Guide provides a variety of processes, 
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resources, and graphic tools to engage all stakeholders 
in developing deeper, sustainable change in schools. 
The Washington SEA, in partnership with the Wash- 
ington Association of School Administrators, produced 
a companion document for districts, School System 
Improvement Resource Guide, to assist districts with 
their improvement efforts. 

Washington enters into a two-year Performance Agree- 
ment contract with identified schools that shows how 
the school, school district, and the SEA will sup- 
port the successful implementation of the school’s 
improvement plan. Prior to the development of this 
plan for improvement, the district, school staff, and the 
SEA consider ways in which the district can support 
the school and staff during the planning process. The 
agreement is organized around a template of 30 district 
commitments and 8 state commitments. The template 
serves as a guide for dialogue between the district and 
the SEA. Participants consider types of evidence that 
would demonstrate support for each of these agreed- 
upon commitments. One example of a district commit- 
ment is: The district agrees to provide supplemental 
funding for specific professional development activi- 
ties for instructional staff based upon strategies identi- 
fied in the school improvement plan. 

Washington uses external facilitators who are paid 
by the SEA to provide direct services to schools. For 
example, School and District Improvement Facilitators 
assist local educators to implement the improvement 
process (including a needs assessment and education 
audit). They help craft the Performance Agreements 
mentioned above as well as the subsequent revision of 
local improvement plans. While they mentor princi- 
pals, the plan is to build capacity and not reliance. The 
Facilitators have no authority, but they carry signifi- 
cant responsibilities as external change agents. They 
work with the SEA, the district, and a School Improve- 
ment Leadership Team to develop and revise a current 
plan to address identified needs and prepare and imple- 
ment the jointly developed Performance Agreement. 
They help the school staff identify and eliminate barri- 
ers to change and promote necessary school consensus 
for change. 

Among the evidence showing the effect of Washing- 
ton’s process are outcomes from several evaluations. 
One finding indicated that fidelity of implementation 
of school improvement efforts has been strong for each 
of the three cohorts of schools that have gone through 
the Performance Agreement process using the Facili- 
tators. Another is that student achievement increased 


during their three years as measured by the percent of 
students meeting the standards on the Washington state 
reading and math tests. 

1.8 Providing differentiated services and resources 
to support district and school improvement 

The performance levels for this indicator range from 
the SEA having no menu of available services and 
resources aligned to a diagnosis of the district to the 
SEA having an ongoing process of evaluating and 
improving its differentiated and aligned services and 
resources to support district/school improvement. 

Ohio’s policy is to use a Tri-Tier model of school 
improvement support to deliver differentiated techni- 
cal assistance based upon need to all districts with 
the greatest emphasis on the districts that have been 
identified by the SEA as being of the highest priority 
(Tier 1). 

The Tri-Tier Model requires districts to go through 
a cycle. They begin by identifying critical needs and 
developing a focused plan; district personnel must 
then implement their plan and demonstrate that they 
have monitored their improvement process. These Tier 
1 districts receive support through 1 6 State Support 
Teams (SSTs) that are located in regions throughout 
the state. Each SST uses the single school improve- 
ment plan and process developed by Ohio in their 
work with the districts. 

Ohio has narrowed the scope and prioritized the 
SST work. Previously, the professional development 
offered by the SSTs lacked a coherent focus and varied 
from region to region. Each SST is responsible for 
implementing a tiered-service delivery model iden- 
tified in the goals and strategies articulated in the 
Performance Agreement between the ODE and each 
SST regional fiscal agent. Among many responsibili- 
ties, the SSTs must focus on developing the capacity of 
Tier 1 district leaders to develop and implement plans 
for effective school improvement systems around the 
themes of leadership, curriculum, instruction, assess- 
ment, and school climate. Subsequently, the state 
evaluates the effectiveness of its intervention policies 
and procedures. 

1.9 Intervening in districts and schools that 
repeatedly do not meet targets for student 
achievement and graduation rates 

The performance levels for this indicator range from 
the state not having a policy authorizing the SEA 
to directly intervene with districts/schools based on 
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persistently unsatisfactory achievement and graduation 
rates to the SEA having an on-going process in place 
to evaluate the effectiveness of its intervention policy. 
Since School Improvement Grant (SIG) schools are 
institutions that have persistently low performance, the 
example below should be useful. 

In Lead Turnaround Partners: How the Emerging 
Marketplace of Lead Turnaround Partners is Chang- 
ing School Improvement, Corbett (2011) describes 
how Lead Turnaround Partners (LTPs) are working 
with SEAs to successfully implement SIG. She notes 
that “Persistently low-achieving schools are inevitably 
a reflection of their districts, and any improvements 
made at the school level are not sustainable without 
systemic changes and improvements” (p.5). For exam- 
ple, Virginia’s SEA created a more involved state role 
when it became clear that LTPs, districts, and schools 
were all trying to figure out how to successfully do 
this work simultaneously but separately. As a result, 
the C ommonwealth sponsors a series of trainings, 
webinars, conference calls, and technical assistance 
sessions. Attendance of the LTP, the district liaison, 
and the school principal is required. LTPs report that 
attending trainings with the district and school staff is 
helpful and ensures that everyone is on the same page. 

Virginia also developed a new position to support the 
improvement efforts. Five state facilitators were hired 
to act as the main liaisons between the districts and the 
state. These state facilitators monitor progress and are 
able to problem solve with the local teams as issues 
arise. The state facilitators share common issues across 
the state. Since the state learns about issues as they 
emerge, staff members are able to resolve any prob- 
lems in a timely ma nn er 2 (Corbett, 2010). 

2. Supports and interventions for all students and 

subgroups 

2.1 Flelping schools and districts better serve 
students with disabilities 

The performance levels for this indicator range from 
no formal state plan for assisting districts/schools 
whose students with disabilities fail to meet achieve- 
ment targets to a fully implemented plan for assisting 

2 See “Condistions for Grant Award, School Division Support” 
on page 4 of Corbett, J. (2010, October, 30). The Virginia story: 
Forging strong working relationships among the state, district, 
school, and external lead partner for the implementation of school 
improvement grants. Lincoln, IL: Academic Development Institute/ 
Center on Innovation & Improvement. 


these districts/schools whose students with disabilities 
do not meet achievement targets. 

Through a Personnel Development Grant, Ohio is 
providing an opportunity to test the integration of its 
special education and general education improve- 
ment models. Ohio leaders believe that the best way 
to make academic gains for all students is to ensure 
a high quality educational system in which all stu- 
dents participate. Therefore, the SEA integrated its 
two existing support systems into unified State Sup- 
port Teams (SSTs). A priority goal of these SSTs 
was to ensure that special education students would 
have access to teachers who understood the academic 
content standards and curriculum implications as well 
as regular classroom teachers. Ohio combined federal 
Title VI-B special education discretionary funds with 
state general revenue funds to provide a single system 
of support to address the needs of all students. 

2.2 Coordinating services for students with 
disabilities across SEA departments and 
programs to maximize service and reduce 
duplication 

For this indicator, the cells range from no state plan for 
integrating its district/school assistance policies to the 
state having fully implemented a plan for integrating 
its district assistance policies, regardless of funding 
sources, to better serve students with disabilities. 

Ohio worked with federal officials to obtain maximum 
flexibility in using federal funds as a support system 
for all students. Ohio modeled how cooperation 
between special education and school improvement 
staff can be accomplished so that funds and require- 
ments can be integrated. This caused school personnel 
to think about how they can most effectively com- 
bine funds for improvement as well. This particular 
approach originally concerned parent advocates who 
were worried that special education services might 
be lost. The SEA maintained that a student-centered 
approach, rather than a programmatic or funding-cen- 
tered approach, would be the best for children. 

The Ohio SEA brought all its departments together 
instead of operating in silos. The goal was to blend 
special education and regular education services 
together in a unified effort to say “kids are kids,” and 
the SEA is going to make sure they are all served. One 
outcome of this process is a single point of contact at 
each SST who is the focus of one-stop-shopping for 
local personnel about school improvement issues. 
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In Delaware, “Special education staff is core to our 
SRAS. Schools under improvement have first priority 
for services provided through SRAS. We include in 
this prioritization all of the special education indica- 
tors. Our application, evaluation, and monitoring 
systems are being revamped so they provide the infra- 
structure to support the SRAS. We revised the district 
and school improvement Success Plans. They articu- 
late the completed strategic plan for the agency — one 
plan that encompasses all students.” 

In Maryland, “The strength of our system is the inclu- 
sion and coordination of cross-divisional and external 
agency support and expertise, including special educa- 
tion. The first step in the process is the review and 
triangulation of various needs assessments to identify 
pervasive as well as isolated needs.. . .The system 
works to build foundational strength in core areas of 
needs and provide enrichment support in more focused 
areas, such as special education. Special education 
staff will have a critical role to play in both areas 
(foundational and enrichment) and are inextricably 
linked in structure and delivery to this system.” 

2.3 Helping schools and districts better serve 
English language learners 

The performance levels for this indicator range from 
no existing formal plan to assist districts/schools 
whose English language learners fail to meet achieve- 
ment targets to full implementation of a plan for assist- 
ing districts/schools whose ELL pupils fail to meet 
achievement targets. 

The Pennsylvania Department of Education encour- 
ages schools to share successful practices through pre- 
sentations at state and local workshops. For example, 
teams from 27 schools gathered for a morning session 
to hear presentations from two schools on how they 
raised achievement for ELLs over a four-year period. 
In the afternoon, members of the presenting schools 
worked alongside the other 27 teams to practice using 
diagnostic screening tools and rethinking intervention 
strategies. Afterward, the Pennsylvania SEA posted a 
summary of the strategies presented on their website. 

2.4 Coordinating services for English language 
learners across SEA departments and programs 
to maximize service and reduce duplication 

The performance levels for this indicator range from 
no formal state plan for integrating district/school 
assistance policies and programs to better serve ELL 
pupils to the full implementation of such a plan. 


In Georgia, Idaho, Iowa, Kansas, and North Carolina, 
the ELL personnel work closely with school improve- 
ment staff to provide regional services and to ensure 
that improvement teams are knowledgeable about best 
practices in this area. 

In New Mexico, the ELL personnel are li nk ed in two 
ways. The first is through participation on a cross- 
functional professional development work group. 

The work group is focused on coordinating both the 
needs and the provision of professional development 
for schools and districts. The second way is through 
close collaboration with the Assessment and Account- 
ability staff and linking the federal Annual Measurable 
Achievement Objectives with the progress of schools 
toward AYP goals. 

In Washington, the Migrant/Bilingual Education staff 
work closely with other units to guide districts whose 
ELLs are not making AYP. The SEA provides districts 
with technical assistance in evaluating their current 
ELL service delivery plans, using data to make adjust- 
ments to their plans, and in identifying necessary 
professional development. This technical assistance 
is often the result of collaboration among the differ- 
ent units at the state level including Title I, special 
education, assessment, and school improvement staff. 
ELL personnel participate in the efforts of the SRAS 
to update school improvement rubrics for districts to 
ensure that the needs of culturally and linguistically 
diverse learners are addressed. 

In Rhode Island, District Corrective Action Plans 
and District Negotiated Agreements contain plans 
that delineate SEA services including ELL for the 
designated district. Each district is provided a Joint 
Capacity Team, which includes members representing 
general education, special education, and ELL staff 
from both the SEA and LEA levels. 

3. SRAS evaluation design 

3.1 Documenting district/school activities provided 
through SRAS 

The performance levels for this indicator range from 
there being no written plan to document SRAS work 
with districts and schools to the level where the SEA 
has a fully operational system for documenting and 
evaluating the effectiveness of its SRAS work. 

The Ohio SEA has an Office of Field Relations that 
collaborates with other offices within the SEA and 
various regional providers to coordinate a Statewide 
System of School Improvement Support. Using the 
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Tri-Tier Model, Ohio provides aligned resources, 
information, tools, professional development, and 
technical assistance to all districts with the greatest 
intensity to those districts in Tier 1 . 

Each of the State Support Teams (SSTs) is responsible 
for implementing a tiered-service delivery model 
identified in the goals and strategies articulated in the 
Performance Agreement between the Ohio SEA and 
the fiscal agents that manage the SSTs. This model 
outlines how all districts are serviced through a differ- 
entiated technical assistance structure with high need 
districts receiving the greatest intensity of service. 

Stage 1 of the Ohio Improvement Process begins 
with a comprehensive assessment of student data and 
identifies academic weaknesses that must be addressed 
using the Decision Framework tool. This decision- 
making process is designed to assist districts using a 
state-developed data warehouse in making informed 
decisions about where to spend their time, energy, and 
resources in order to make significant and substantial 
improvements in student performance. The SSTs work 
with these districts through all four stages of the Ohio 
Improvement Process, including the last one in which 
everyone monitors and documents the impact of the 
implemented improvements. In fact, Ohio’s SSTs 
perform quarterly Partnership Agreement reviews with 
all Tier 1 districts to document evidence of implemen- 
tation and impact and to revise strategies as needed 
based upon data. An annual summary of technical 
assistance provided to each Tier 1 district and the 
impact of those services on the district is provided by 
each SST in June. This summary captures the district’s 
efforts and changes in practices and helps guide ongo- 
ing work while documenting history and progress. 
These results are reviewed at the regional and state 
levels. 

3.2 Evaluating the SRAS 

The performance levels for this indicator range from 
no written evaluation policies and procedures to 
determine the effectiveness of the SRAS to the SEA’s 
evidence that it has used the results of the evaluation 
process to improve its SRAS. 

Alabama evaluates their SRAS based on the analyses 
of local personnel about the effectiveness of the col- 
laborate efforts of the state support teams. In particu- 
lar, it evaluates peer mentors and Regional School 
Improvement Coaches (RSIC) at mid-year and at year 
end. This appraisal presents task statements, such 


as whether the RSIC personnel are providing effec- 
tive on-site support, coaching, and guidance to local 
personnel. Evidence is requested to support ratings. 
Finally, there are “listening post” opportunities offered 
to all superintendents for voicing concerns as well 
as positive comments at regional meetings with the 
Deputy Superintendent and key staff. 

The Ohio SEA collects customer satisfaction data. 
Techniques include surveys, interviews, and portfolios 
for individual districts that include a history of what 
services the SSTs provide. 

In addition to independent evaluations, the state of 
Washington works with the Center for Educational 
Effectiveness, which annually provides a perception 
survey, entitled the Educational Effectiveness Survey. 

Edvantia evaluated Tennessee’s Exemplary Educator 
(EE) program based on document reviews, surveys, 
and achievement data. As part of the evaluation, Ten- 
nessee established a system in which each EE com- 
pletes an end-of-year status report for each school or 
district assisted. Edvantia staff examined and evaluated 
these status reports to determine progress in meeting 
performance expectations and to identify the broadest 
areas of need for the high priority schools and districts 
as well as the actions of the EEs. At the same time, the 
EEs evaluated the tools and training provided to them. 

In all these states, summative evaluation criteria 
include student achievement outcomes. 

3.3 Evaluating the SEA assessment program 
The performance levels for this indicator range from 
the SEA having no policy in place to evaluate the 
effectiveness of its assessment program to evidence 
that the SEA systematically evaluates its assessment 
program to assure the rigor of its test yields reliable 
and valid inteipretations. 

In order to give credibility to its new, more rigor- 
ous standards, Tennessee notes in its 2012 flexibility 
waiver request that it revamped its state assessment 
system to provide a more accurate indicator of stu- 
dent performance, including resetting its cut scores to 
more closely align with national standards for NAEP 
and the ACT. Tennessee acknowledged using inflated 
scores for years but touted its new standards and 
more demanding graduation requirements as the path 
toward raising expectations for all students. By way of 
example, the percentage of students scoring proficient 
or advanced on the 7th grade math test dropped from 
90.3% in 2009 to 28.5% in 2010 (Tennessee, 2012). 
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Florida noted in its 2012 flexibility waiver that it 
recently modified its Florida Comprehensive Achieve- 
ment Test (FCAT) to make it a value-added model for 
measuring student growth. This process for measuring 
student learning growth is being used in all district 
teacher and principal evaluation systems during the 
2011-2012 school year. In addition, Florida believes its 
own value-added model lays the foundation for a new 
way of measuring student growth. 

Florida is also addressing the issue of what has 
become known as “non-tested” grades and subjects 
by developing a statewide item bank in all grades and 
subjects. The item bank will include items for core 
courses in grades K-12 and Spanish with software to 
facilitate high quality test development as well as a 
vetting process to ensure the items themselves are high 
quality and aligned with Common Core State Stan- 
dards (Florida, 2012). 

Part B: Resources 

4. District and school staff needs 

4.1 Enhancing the supply of teachers and 
leadership personnel skilled in school 
improvement strategies 

The performance levels for this indicator range from 
the state not having any formal strategies to enhance 
the supply of teachers and administrators skilled in 
school improvement to the state having evidence that it 
has increased the supply of educators skilled in school 
improvement. 

Kentucky has an agreement with the University of 
Kentucky to provide leadership certification for its 
Highly Skilled Educators (HSEs) through the comple- 
tion of three courses. This program was designed to 
select and reward the most outstanding teachers and 
administrators with recognition for excellence, a salary 
incentive, and an opportunity to assist other teachers, 
administrators, and schools. Schools whose scores 
declined have been assigned an HSE. A key element 
of this program is that the HSEs remain employees of 
their home district and after their tenure as a HSE, they 
return to their district and become a resource to build 
internal district capacity. Of the 1 05 HSEs who entered 
the program as teachers, 8 1 accepted leadership posi- 
tions beyond that of classroom teacher after exiting the 
program. 

In addition, the Kentucky State Department joined 
with the Kentucky Association of School Adminis- 
trators to start the Kentucky Leadership Academy 


to provide an opportunity for Kentucky educational 
leaders, at all levels of leadership skill and develop- 
ment, to receive the necessary support to assist them in 
positively impacting whole school improvement and 
advancing student learning though ongoing profes- 
sional growth opportunities. The core values of the 
Leadership Academy include: effective leaders have 
high expectations for themselves, their staff, and stu- 
dents; high expectations are evidenced in the work of 
the district/school and in student achievement; and that 
highly effective leaders recognize and communicate 
the district’s/school’s deficiencies and collaboratively 
work for improvement with the learning community. 

Ohio developed eLearning modules to standardize 
professional development. These modules are designed 
to strengthen instructional strategies and are focused 
toward schools that demonstrate the greatest need. 

The goal is to link professional development to data 
and then offer customized professional development 
opportunities via these modules. Ohio believes that 
eLearning is an effective way to disseminate consis- 
tently high-quality professional development content 
statewide. 

In its 2012 flexibility waiver proposal, Florida dis- 
cusses its plan to prepare aspiring school leaders to 
effectively address the teaching and learning chal- 
lenges of chronically low-achieving high schools 
and their feeder patterns. The primary objective is to 
create a pool of the most promising candidates that can 
turn around schools through an innovative, problem 
solving-based program of study. The objective will 
be achieved by working with seven districts to recruit 
and train a minimum of 80 to 100 new principals and 
assistant principals to serve in Florida’s persistently 
lowest-achieving schools (Florida, 2012). 

4.2 Providing incentives and strategies for 
addressing a more equitable distribution of 
well qualified and effective teachers within and 
across districts 

The performance levels for this indicator range from 
the SEA having no plan to address the quality of teach- 
ers in schools identified as needing improvement to the 
state having systematically implemented and evalu- 
ated its plan to achieve more equitable distribution of 
highly qualified elementary and high school teachers 
within and across districts. 

The Ohio Department of Education developed a 
Teacher Distribution File (TDF) in response to its own 
research which showed that often the children in the 
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low-performing schools who need the most experi- 
enced, educated, and skilled teachers instead had the 
least effective educators. School districts use the TDF 
to determine where their teacher inequities exist. The 
TDF can: 

• Identify the percentage of minority and economi- 
cally disadvantaged students who are taught core 
subjects by inexperienced (less than three years’ 
experience) teachers vs experienced teachers; 

• Identify by core subject area and by school 
where more that 10% of the core subject courses 
in schools are taught by teachers who are not 
highly qualified; 

• Identify the percentage of inexperienced vs 
experienced teachers in high-poverty schools vs 
low-poverty schools; and 

• Identify the percentage of highly effective vs 
non-effective teaches in high-poverty schools vs 
low poverty schools. 

After conducting these analyses, Ohio school district 
personnel write their District Teacher Equity plans. 

As part of its statewide equity plan, Ohio is develop- 
ing a system in which it will continuously monitor and 
improve the distribution patterns of Ohio’s teachers to 
ensure that poor and minority students are not being 
taught at higher rates than other students by inexperi- 
enced, unqualified, or out-of-field teachers. 

4.3 Recruiting and retaining well-qualified and 
effective teachers 

The performance levels for this indicator range from 
the state has no plan to enhance recruitment and 
retention of highly effective teachers to the state has 
a fully implemented plan to enhance recruitment, as 
well as evidence that recruitment and retention have 
improved. 

Ohio has the Council Attracting Prospective Educa- 
tors (CAPE) to identify and attract young, talented 
people from diverse backgrounds to a career in teach- 
ing with the goal of increasing minority representation 
in teaching. The CAPE Teacher Academy is a five-day 
summer academy experience designed to introduce 
Ohio high school students to teaching as a career 
possibility. The academy provides an opportunity for 
50 high school students from diverse backgrounds to 
experience a university environment, develop leader- 
ship skills, explore a career in education, and interact 
with peers and professional role models. 


In addition, there is the Ohio Future Educators Asso- 
ciation (OFEA), a statewide organization for middle 
and high school youth who are interested in a career in 
education. Along with Phi Delta Kappa (PDK), OFEA 
works with advisors and officers of local PDK chap- 
ters to recruit members, plan projects, share program 
information, and provide a statewide communication 
network to motivate students who are interested in a 
career in education. OFEA encourages students to set 
educational career goals early in life, focus on aca- 
demic achievement, explore teaching through direct 
experience in the classrooms, and become citizen lead- 
ers through school/community service. 

However, the major Ohio program for this indica- 
tor is its Career Lattice Framework. This Framework 
expands teacher leadership opportunities and drives 
collaboration between teachers and administrators on 
school design, leadership, and school policy. The goal 
of the Framework is to create a common culture of 
teacher professionalism, improve teacher retention, 
and ultimately enhance student achievement. It also 
provides a framework for teachers to create and sustain 
a community of professional practice where they have 
opportunities to collectively reflect upon their teach- 
ing, consider the progress their students are making, 
learn about and apply new knowledge in their fields, 
and help each other improve. 

4.4 Recruiting and retaining effective district and 
school leadership personnel 
The performance levels for this indicator range from 
the state has no plan to enhance recruitment and 
retention of highly effective school leaders to the state 
has a fully implemented plan to enhance recruitment 
with evidence that recruitment and retention have 
improved. 

Through the Pennsylvania Inspired Leadership pro- 
gram (PIL), Pennsylvania made a large effort to train 
every principal in the state so principals of all schools 
are more effective, not just those leading schools 
in need of improvement. Every novice principal is 
required to take a principal induction program within 
the first five years of service. The induction program 
must address the three core standards: 

• The leader has the knowledge and skills to think 
and plan strategically, creating an organizational 
vision around personalized student success. 

• The leader has an understanding of standards- 
based systems theory and design and the ability 
to transfer that knowledge to the leader’s job as 
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the architect of standards-based reform in the 
school. 

• The leader has the ability to access and use 

appropriate data to inform decision-making at all 
levels of the system. 

This training models the behaviors it wants leaders 
to foster in schools and districts. Participants discuss 
concrete problems of practice in their jobs and are sup- 
ported by coaching and expert modeling in applying 
the content towards solutions in their daily work. The 
Pennsylvania Department’s goal was to create com- 
prehensive standards-based continuing professional 
education programs designed around what the research 
says is good professional development. Participants 
complete evaluations after each P1L training. A local 
university analyzes these evaluations and uses them to 
produce an annual summative report. 

4.5 Engaging institutions of higher education 
(IHEs) to better prepare new teachers and 
leadership personnel 

The performance levels for this indicator range from 
the SEA having no written policies with IHEs to 
jointly ensure that teacher and administrator leadership 
programs adequately prepare their students to the SEA 
and IHEs having an ongoing process for evaluating 
and improving the effectiveness of their policies for 
jointly ensuring that future educators understand state 
standards, curricula, assessments, and effective profes- 
sional practice. 

Ohio requires both public and private teacher prepara- 
tion programs to provide pre-service instruction on 
the state’s accountability system, including standards 
and assessments. The state also requires school leader 
preparation programs to provide pre-service instruc- 
tion for school leaders on the state’s accountability 
system. 

Alabama has an official connection with teacher 
pre-service institutions that require IHE’s to assure 
that teachers know and are able to implement state 
initiatives. Universities receive feedback from novice 
teacher evaluations with results published in a report 
card. There are consequences to the institution if the 
graduates do not perform at the established standard. 

In its 2012 flexibility waiver request, Tennessee noted 
that by the 2014-2015 school year, all new public 
school teachers and principals who received train- 
ing at Tennessee institutions of higher education will 
be prepared to teach the CCSS (Common Core State 


Standards). The state will also revise its licensure 
requirements by: requiring new teachers and principal 
candidates to demonstrate mastery of CCSS content 
through a skills assessment or portfolio project; updat- 
ing reciprocation procedures to ensure that out-of-state 
teachers wishing to gain Tennessee licensure have 
received appropriate training in CCSS content; requir- 
ing teachers entering the school system through alter- 
native certification pathways to be trained in CCSS 
content (Tennessee, 2012). 

4.6 Providing guidelines for the evaluation of 
teachers and principals 

The performance levels for this indicator range from 
the SEA not having evidence that stakeholders had 
an opportunity to comment about the development of 
SEA guidelines for teacher and leadership evaluations 
to the SEA having an ongoing process to evaluate the 
effectiveness of its policies for assisting districts and 
schools with staff evaluation. 

In July 2011, Tennessee became one of the first 
states in the country to implement a comprehensive 
student outcomes-based state-wide educator evalu- 
ation system. The Tennessee Educator Acceleration 
Model (TEAM) is a comprehensive evaluation tool 
designed to improve instructional practices through 
frequent observation of teachers and principals. Under 
the TEAM model, 50% of the educator’s final effec- 
tiveness rating is based on observations conducted 
by trained officials; 35% of the rating is based on 
a student growth measure; and 15% of the rating is 
based on an achievement measure that is coopera- 
tively agreed upon between the educator and evalua- 
tor (Tennessee, 2012). Tennessee continues to make 
adjustments to the TEAM evaluation model through 
their structured process for gathering feedback. Staff 
members have already met with nearly 5,000 educators 
across the state; a third party process is also collecting 
substantial feedback from stakeholders. 

Tennessee also convened twelve development teams 
of teachers and content specialists in the non-tested 
grades and subject areas to make recommendations 
around alternative growth measures in the new teacher 
evaluation system. As a result, four observational 
models of teacher evaluation were developed and 
piloted (Tennessee, 2012). 

In Ohio, legislation required standards for teach- 
ers, principals, and professional development. These 
standards guide training, provide a tool for developing 
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coaching and mentoring programs, and support higher 
education in developing and providing content and 
requirements for pre-service training and ongoing 
professional development. 

Florida documents in its 2012 flexibility waiver 
request that when it won the Race to the Top grant, 
it developed and issued specific guidelines for dis- 
tricts for developing teacher and principal evaluation 
systems. These guidelines provide the criteria for 
how participating districts substantiate that their new 
teacher evaluation systems meet all requirements of 
existing Florida law (Florida, 2012). 

5. Funding of improvement efforts 

5.1 Coordinating state and federal funding streams 
and programs 

The performance levels for this indicator range from 
the state’s SRAS having no plan to efficiently coor- 
dinate programs with different funding sources that 
are aimed at improving schools to fully implement- 
ing a program to integrate planning and documenting 
evidence of greater efficiency in planning resource 
allocation and compliance monitoring. 

One of Ohio’s main strategies was to integrate a 
variety of state and federal programs for seamless and 
efficient delivery of services. The Ohio Comprehen- 
sive Continuous Improvement Plan moved districts 
from multiple program requirements for improve- 
ment planning and compliance into one planning tool 
where resources are aligned with needs and plans of 
actions. Prior to the CCIP, schools were required to 
create separate plans for each of their state and fed- 
eral programs. The CCIP integrated the many plans 
into a single document aligned with fiscal resources 
that serve as a guide to districts and schools in their 
improvement efforts. 

When asked, “What are some lessons learned from 
your state’s experience with a statewide system of sup- 
port that would be helpful to other states?” Ohio staff 
responded that they are now using a student-centered 
rather than a program-centered approach. They began 
to model internally how cooperation among depart- 
ments can be accomplished so that funds and require- 
ments can be integrated. Among the first goals was to 
integrate special education with school improvement 
activities. Their philosophy was that by improving the 
whole system, they improved the learning of special 
education students. 

Ohio combined federal Title VI-B special education 
discretionary funds with state general revenue funds to 


provide a single system of support that addressed the 
needs of all students. 

While Rtl is a federal special education initiative, 
Pennsylvania is defining it as an instructional pro- 
gram beneficial to all students. All students, including 
students with disabilities, are to be given high-quality 
instruction against the same standards, and all stu- 
dents are administered the same progress monitoring 
and benchmark assessments. As a result, Rtl in Penn- 
sylvania is considered fundamentally about school 
improvement. 

5.2 Assisting districts in assessing their use of 
financial resources to fund improvement efforts 
The performance levels for this indicator range from 
no clearly defined process in place to help districts 
analyze their budgets to align financial resources with 
identified needs to the state showing that its budget 
staff systematically is part of the monitoring process to 
help local staff with these fiscal problems. 

Ohio has school finance staff join school improve- 
ment personnel in their State Diagnostic Team on- 
site review process. The philosophy is that everyone 
needs to know how the money is being spent. Each 
team reviews low-performing schools to ensure that 
their budgets are aligned with identified needs. As a 
team, they work together to ascertain how schools 
can become more effective with their programs and 
more efficient with their financial resources. When 
these school improvement teams develop a list of 
recommendations, the finance representatives can 
immediately help the districts identify which funds are 
available to implement the recommendations, and they 
can also point out which recommendations are too 
costly to start immediately. 

6. Data analysis and use 

6. 1 Providing a comprehensive SEA data system 
The performance levels for this indicator range from 
the SEA having a data system that meets minimum 
reporting requirements to one that provides timely, 
accurate, and integrated data that reduces redundancy 
and is useful for districts/schools to inform improve- 
ment decisions. 

Pennsylvania established a permanent statewide stu- 
dent identifier for its 1.8 million Pre-K through high 
school students. In addition, all teachers and certified 
staff were assigned Professional Personnel Identi- 
fiers. By fall 2008, the SEA had integrated data from 
seven of its eight former stand-alone databases so it 
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now collects data on 72 unique elements, including 
extensive student and staff information, course enroll- 
ment, and attendance. The state department can now 
determine the proportions of highly effective teachers 
from one district to another. The Pennsylvania Depart- 
ment of Education now possesses the ability to answer 
a host of questions that before were left to guesswork. 
The state continues to build a longitudinal database to 
ascertain whether its investment in numerous programs 
is working. 

Tennessee noted in its 2012 flexibility waiver that it is 
currently developing robust data systems which will 
allow teachers, schools, districts, and the state to track 
and learn from student progress and other indicators 
at each level. Their primary focus is on teacher evalu- 
ation, a more robust student information system, an 
expanded Tennessee Value Added Assessment System, 
and a P-20 statewide longitudinal data system. The 
data systems will allow the state to monitor the ways 
in which CCSS (Common Core State Standards) 
instruction drives student progress, learn from the 
CCSS-aligned field test items how well students are 
achieving the standards, and study the extent to which 
teachers are delivering CCSS quality instruction (Ten- 
nessee, 2012). 

6.2 Using assessment data 
The performance levels for this indicator range from 
the state providing minimal guidance to educators on 
the use of assessment data in their improvement plan- 
ning to the SEA having an ongoing policy and process 
for assisting education personnel in how to effectively 
use assessment data in their improvement planning. 

Ohio’s SSTs devote a considerable amount of time 
helping district staff understand assessment data and 
how to use it for improvement planning. Local per- 
sonnel can access item analysis data by classroom. 

The Ohio Department of Education staff believes 
improvement is highly focused, begins with an honest 
assessment of student data, and identifies academic 
weaknesses that must be addressed. A state-developed 
data warehouse allows relevant data to be readily 
available to districts and buildings. These data are 
organized in such a way as to allow district and school 
leadership teams to answer essential questions and 
make decisions about their greatest needs related to 
improving student performance. 

Data are examined in relation to student performance 
in content areas, identifying the weakest grade levels, 


subgroups with poor performance, and the extent to 
which the problems are present throughout the district. 
Once the student performance needs are identified, 
the district then looks at performance in relation to 
instructional management (curriculum, instruction, and 
assessment), professional development, and educator 
quality. 

7. Support teams and school improvement 

consultants 

7. 1 Matching districts/schools with support teams 
and school improvement consultants 

The performance levels for this indicator range from 
the state not having a systematic procedure in place 
to match schools and districts with consultants based 
upon qualifications and identified needs to the state 
using data to evaluate these improvement consultants 
and support teams as well as the state having evidence 
that these experts have had a positive impact. 

Alabama has a system of peer mentors and regional 
school improvement coaches (RSIC). Peer mentors 
serve schools in year four or more of school improve- 
ment status. The RSICs work with districts on a 
regional basis. Members of the Alabama Roundtable 
annually select these change agents. Many of them are 
already employees of the districts in school improve- 
ment status. They are selected for special year-long 
training, so they can help their own districts and others 
in need of support. After their time in this role has 
finished, most go back to full-time status with their 
own districts. In this way, the capacity of the district 
has been enhanced. 

These Alabama peer mentors work full time and are 
assigned to no more than two schools. Elowever, the 
goal is clear that the state expects these mentors to also 
build capacity at the district level so central office staff 
can write better plans for improvement. “Change at the 
school level requires continuous district support.” 

7.2 Training, supervising, and evaluating support 
teams and district/school improvement 
consultants 

The performance levels for this indicator range from 
no documented plan for training, supervising, and 
evaluating support teams/improvement consultants 
to the existence of a comprehensive training program 
that addresses identified needs along the way and is 
formatively revised. 

In Tennessee, the key providers of technical assistance 
to schools and districts are the exemplary educators 
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(EEs). In high-priority schools and districts, EEs 
provide support in school improvement planning, use 
of data, curriculum, and effective instructional and 
organizational practices. Tennessee contracts with an 
external firm to hire the EEs and provide their training. 
The EEs are trained during the summer and for five 
weeks throughout the year to keep them current. The 
consultants are assigned based on the identified needs 
of schools or districts and the expertise of the EE. 

The training enables the EEs to provide differentiated 
services. 

8. External partners and providers 

8.1 Managing and coordinating organizational 
partners 

The performance levels for this indicator range from 
having no evidence that the state has formed partner- 
ships with other organizations to further the goals 
of the SRAS to evidence that external partnerships 
have been actively implemented and resulted in 
increased resources and services for school and district 
improvement. 

Ohio’s SSTs involve various partners in the review of 
data and identification of needs. These partners may 
include, but are not limited to: institutions of higher 
education, Ohio Education Service Centers, and Infor- 
mation Technology Centers. Partners provide services 
based upon their existing relationships or content/pro- 
cess knowledge expertise. 

In Oklahoma, university staff participates on the 
School Support Teams that make site visits, observe 
classrooms, provide feedback to teachers and admin- 
istrators, and interview parents, students, teachers, 
and administrators. They also provide feedback to 
School Support Team Leaders for reports to building 
administrators. 

In Pennsylvania, university personnel are part of con- 
tent expert teams; in North Carolina, university per- 
sonnel deliver professional development to turnaround 
administrators. In Missouri, university personnel pro- 
vide research capabilities and statistical data analyses 
for their student achievement data and demographic 
information. 

Kentucky identified its state’s education associa- 
tions as critical in the evolution of its system. Ken- 
tucky works with over 60 agencies to support school 
improvement. One example is the Partnership for 
Successful Schools. This Partnership is a coalition 
of Kentucky businesses, educational groups, and 


government officials who believe that business has 
both a stake and role to play in helping children learn. 
The Kentucky Department of Education is a leading 
member of the Educational Leadership Development 
Collaborative, an association of educational organiza- 
tions in Kentucky working together to improve student 
learning through leadership. The group’s mission is “to 
advance student learning through a collaborative focus 
on leadership development.” 

8.2 Providing guidance for tutoring and extended- 
learning time 

The performance levels range from having no clearly 
defined procedure to assist districts/schools in provid- 
ing tutoring and extended learning time for students to 
the SEA having an ongoing process for evaluating and 
improving the efficiency and effectiveness of its poli- 
cies for assisting districts/school in providing tutoring 
and extended learning time. 

In its 2012 flexibility waiver request, Florida notes 
that it strongly believes in extending the instruc- 
tional day, week, and year as a strategy to increase 
student achievement. Florida provides Supplemen- 
tal Academic Intervention (SAI) funding based on 
the number of students needing an extended school 
year program. These funds are provided to all LEAs 
prior to the beginning of each school year allowing 
schools to establish academic intervention programs 
at the moment students begin to struggle with subject 
content. In addition to SAI funds, schools have access 
to school improvement and Title I funds to extend 
instructional time (Florida, 2012). 

Part C: Implementation 

9. Removal of barriers to change and innovation 

9. 1 Removing barriers to change 
The performance levels for this indicator range from 
the state having no process or efforts underway to 
identify and remove barriers to change to evidence 
that districts and schools have availed themselves of 
increased opportunities, and the process is constantly 
being evaluated. 

Washington State has a process for districts to request 
waivers from specific state requirements. For example, 
a district may apply for less than the full instructional 
year of 180 days after the community approves. The 
days are often used for professional development. Dis- 
tricts may request waivers from provisions in teacher 
contracts if a school improvement plan calls for that. 
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Tennessee has established a systematic review process 
to determine what state policies and other barriers 
hinder improvement efforts in districts and schools. 

The SEA has a Project Management Oversight Com- 
mittee that provides the necessary coordination to 
conduct these reviews. Tennessee also provides for 
alternative certification routes. 

Under the Mandate Waiver Program, Pennsylvania law 
allows public school districts, along with vocational- 
technical schools, Intermediate Units, and groups of 
schools to seek exemption from certain provisions of 
the Public School Code. Though regulations surround- 
ing teacher contracts or the certification requirements 
for principals and teachers cannot be waived, appli- 
cants can seek waivers on other matters if they can 
demonstrate one of two things: 1) that a waiver would 
improve instructional programs, or 2) that a waiver 
would result in more efficient, effective, or economical 
operations. 

9.2 Creating options for new types of schools, 
including charter schools 

The performance levels for this indicator range from 
the state taking no action to create options for new 
types of schools to evidence that there are a variety of 
new types of schools, and the SEA is constantly evalu- 
ating the efficiency and effectiveness of its policies. 

In 1 997, Pennsylvania state law authorized the creation 
of charter schools. In 2002, a law passed allowing 
cyber charter schools. The Pennsylvania Department 
of Education is the authorizer for all cyber charter 
schools, but local school boards must authorize brick 
and mortar charter schools. There is no cap on the 
number of charter schools that may be authorized in 
Pennsylvania. 

Ohio allows charter schools to open in areas where 
schools are continually underperforming. However, 
if a charter school is classified as being in a state of 
“academic emergency” for three years, the charter has 
to close. 

In its 2012 flexibility waiver application, Florida 
recognized the role that charter schools can play in 
operating a turnaround school or opening a new school 
within the feeder pattern of a chronically low achiev- 
ing high school, so they created a separate strand 
within their new leadership preparation program (see 
indicator 4.1 for more information). There will be 
leadership training for 20-25 current or aspiring char- 
ter school leaders in Florida’s lowest achieving seven 
districts. The charter school strand will emphasize 


knowledge and behaviors that enable school leaders to 
promote successful teaching and learning, collabora- 
tive decision-making strategies, distributed leadership 
practices, a culture of collegiality in analysis and use 
of data and instructional technologies to guide school 
improvement activities. In addition, the strand will 
focus on the effective use of the flexibility and auton- 
omy provided to charter schools (Florida, 2012). 

9.3 Expanding access to college level courses or 
their prerequisites, dual enrollment courses or 
other accelerated learning opportunities 
The performance levels for this indicator range from 
the SEA not developing policies and procedures to 
work with high schools and IHEs to expand access to 
college level courses to the SEA having an ongoing 
process to evaluate the effectiveness of its policies to 
work with high schools to expand access to college 
level courses or their prerequisites as well as other 
accelerated learning opportunities. 

One proxy for expanding access for high school 
students to college level courses is to increase the 
number of high schools offering AP (Advanced Place- 
ment) tests. (AP has become a gold standard because 
any school can slap the label “honors” on a class but 
AP requires outside validation.) To offer official AP 
courses, teachers and principals must develop a cur- 
riculum that the College Board attests meets standards 
set by college faculty. Many AP teachers also undergo 
special training since AP involvement pushes schools 
and policymakers to talk about raising the “ceiling” 
thus elevating students beyond the bare minimum and 
pushing them toward college. 

Six states now require high schools to offer AP: Arkan- 
sas, Connecticut, Indiana, Louisiana, Mississippi, 
and South Carolina. An additional six states require 
schools to offer either AP or other rigorous classes 
such as dual-enrollment or International Baccalaureate. 

States encourage AP in other ways. Indiana gives 
schools bonuses for AP performance and factors AP 
into the state’s accountability formula and performance 
goals. (Indiana has also established a statewide goal 
of 25% of graduates earning AP credit.) Florida pays 
bonuses to teachers for each student earning a quali- 
fying score. Seven states require public colleges to 
award credit or placement based on AP exam scores. 
Students, meanwhile, usually get extra weighting on 
their GPAs and improved chances for admission to 
selective colleges (Pope, 2012). 
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In its 2012 flexibility waiver request, Tennessee noted 
that it intends to incentivize districts to work with their 
local IHEs to expand postsecondary credit offerings 
and is working to expand dual enrollment, and dual 
credit. Secondary and post-secondary institutions have 
received grants to implement workable articulation, 
dual credit, and dual enrollment opportunities (Tennes- 
see, 2012). 

In its 2012 flexibility waiver request, Florida noted 
that a $50 bonus is earned by an AP teacher for each 
student in each AP course who receives a score of 3 
or higher on the AP examination. An additional bonus 
of $500 is earned by the AP teacher in a school desig- 
nated with a performance grade category “D” or “F” 
who has at least one student scoring 3 or higher on an 
AP exam (Florida, 2012). 

10. Incentives for change 

10.1 Setting consequences for low student 
achievement and low graduation rates 
The performance levels for this indicator range from 
the state using only the required sanctions for low 
performance and/or low graduation rates to evidence 
that consequences for low performance and gradua- 
tion rates are evaluated and subsequently revised and 
improved. 

Alabama’s system of academic standards, assessments, 
and accountability is a single system that applies to all 
LEAs and all schools irrespective of their receipt of 
Title I funds. All LEAs and schools are subject to the 
state’s definition of performance targets for achieve- 
ment and graduation of all students and each subgroup. 
Alabama’s system of sanctions for schools identified 
for improvement adheres to the following principles: 

Sanctions should result in increased learning opportu- 
nities for students. 

• The state’s primary response to schools that are 
not making academic progress should be inten- 
sive support to the instructional programs. 

• Sanctions should establish a priority for state 
support to LEAs as they provide appropriate 
levels of school improvement guidance begin- 
ning in the first year of failure to make perfor- 
mance targets. 

• The magnitude of sanctions should reflect 
the magnitude of the need for academic 
improvement. 

Districts in Alabama with schools that have been on 
the list for needing improvement for four or more 


years must hire a school improvement specialist at the 
district level. 

Ohio places a public spotlight on districts that show 
continued low performance in its federal and state 
blended accountability system. Ohio has five designa- 
tions as well as a federally required special education 
IDEA annual determination based on a district’s ability 
to implement IDEA. 

10.2 Providing positive incentives for 
improvement 

The performance levels for this indicator range from 
no incentives provided for districts or schools for 
improved academic achievement to evidence that 
positive incentives provided by the state promoted 
an increased number of district/school programs and 
conditions for raising student achievement. 

Alabama employs one or both of the following criteria 
to identify schools and LEAs for academic recognition 
and/or rewards: 

• The school or district “significantly” closed the 
achievement gap between subgroups of students 
and made progress toward performance targets. 
“Significantly” is based on year-to-year analy- 
ses of test scores with input from the state’s 
Accountability Advisory Committee. 

• The school or district made its performance tar- 
gets for two or more consecutive years. 

Principals and teachers in Alabama who are highly 
effective and have been instrumental in closing the 
achievement gap and/or successfully making their 
performance targets will be identified to serve as peer 
consultants on State Support Teams. 

A major incentive in Alabama is the selection of 
Torchbearer schools, recognizing high performance 
among Alabama’s high poverty schools. 

In Ohio, there is public recognition for schools that 
show improved results, especially “Schools of Prom- 
ise” that have high achievement and high poverty and 
“Schools of Distinction” that have high achievement 
and a high percentage of special education students. 

In Ohio, legislation provides teachers a $20,000 sign- 
ing bonus or loan forgiveness in return for a commit- 
ment to teach in a hard-to-staff school for at least five 
years. Ohio also has special assistance available to 
districts with high concentrations of poverty. One por- 
tion of this funding is provided to districts where the 
percentage of schools in academic distress is higher 
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than the state average. A district can receive a 3.5% 
increase in funds if they reduce that percentage. 

10.3 Publicly disclosing district and school 
performance 

The performance levels for this indicator range from 
the state disclosing school and district performance 
data only as required to evidence that the reports and 
website are used and helpful to users. Refinements are 
made based on user feedback. 

In Washington State, the school report card, acces- 
sible through the SEA website, gives disaggregated 
state assessment results by subgroup and content area. 
The state’s data carousel process has been instrumen- 
tal in school and district improvement planning. For 
example, when staff at one school wanted to see what 
schools with similar student populations were doing to 
help their students succeed, they were able to use the 
data filters on the website to identify schools that they 
later visited. 

In Tennessee, education personnel interviewed indi- 
cated that the state provides a wealth of data at their 
website that districts and schools can use to develop 
their improvement plans, including value-added data. 
One of the principals describe the value-added data as 
a “wonderful, wonderful tool from the state. . .they’re 
online, they’re accessible. The value-added really 
helps me as a principal to look at individual teacher 
growth.” 

11. SRAS Communications 

11.1 Communicating with clear and systematic 
communication paths within the SRAS 
The performance levels for this indicator range from 
no documented plan for a statewide communication 
system among those who provide support such as state 
education department employees, regional offices, uni- 
versities, and other elements of the SRAS to evidence 
that the communications are being used to improve 
services and support. The state formatively evaluates 
the quality and accuracy of the communication to the 
field and service providers. 

Alabama’s Accountability Roundtable’s mission is to 
provide a coordinated, seamless system of continual 
technical assistance and support to schools in the 
areas of curriculum, instruction, fiscal responsibility, 
management, and leadership. Technical assistance to 
schools is coordinated through this Accountability 
Roundtable and provided by the State Support Team. 
The Accountability Roundtable meets regularly with 


representatives from all departments. Table Talk is a 
bimonthly newsletter disseminated by the Roundtable 
to members of the State Department of Education. 

In Washington State, there are nine regional Edu- 
cational Service Districts (ESDs) across the state. 

They are funded through state, local district funds, 
and grants. The ESDs provide representation at each 
of the monthly meetings of the school improvement 
facilitators. 

Kentucky has a Bureau Leadership Planning Team 
composed of the commissioner, the deputy, and all 
associates. They meet at least monthly to discuss, 
review, and evaluate the statewide system of sup- 
port. Additionally, two key associate commission- 
ers share the responsibility for day-to-day activities 
around statewide support. The SEA relies heavily on 
its approach of cross-agency decision making, maxi- 
mizing limited resources, and shared ownership of 
initiatives and responsibilities. The State Department 
of Education is a leading member of the Educational 
Leadership Development Collaborative, a unique 
association of educational organizations in Kentucky 
working together to improve student learning through 
leadership. The group’s mission is “to advance student 
learning through a collaborative focus on leadership 
development.” This group meets monthly to discuss 
current issues in education and address programs and 
strategies being used by their respective organizations 
to improve student learning. 

1 1.2 Implementing clear and systematic 

communication paths between the SEA/SRAS 
and districts/schools as well as significant others 
The performance levels for this indicator range from 
having no documented procedures for communica- 
tions between the state/SRAS and districts/schools to 
evidence that the communications are being used to 
improve services and support. The state formatively 
evaluates the quality and accuracy of its communica- 
tion to the field and service providers. 

Louisiana brought local superintendents and local 
school boards to the forefront of accountability by 
including them in discussions related to interventions 
in specific schools. “The movement toward increased 
state intervention would be more difficult if the LDE 
did not involve the local superintendent and their 
boards in preparation of possible interventions and 
sanctions.” 

In Tennessee, the nine Field Service Centers assist 
schools and systems in school improvement planning 
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and other functions. The Director of the Field Service 
Centers is based at the State Department of Educa- 
tion and reports to an Assistant Commissioner. This 
enhances the communications with the field. 

In Washington State, local school personnel inter- 
viewed noted that a key resource for the schools and 
districts is the State Department of Education website, 
which they consult for data, professional develop- 
ment materials, grade level expectations, and other 
resources. 

12. Technical assistance 

12.1 Delivering training to districts and schools in 
school improvement planning; implementation, 
and monitoring 

The performance levels for this indicator range from 
the state not having a formal, documented process 
for training and assisting districts/schools in school 
improvement planning to having a plan fully imple- 
mented, the quality of services regularly evaluated, and 
evidence that delivered services have a positive impact 
on school improvement planning and implementation. 

In the State of Washington, the School Improvement 
Planning Process Training Guide is written as a plan- 
ning document that leads the principal and school 
community through a cycle of continuous improve- 
ment. The guide provides a variety of processes, 
resources, and graphic tools to engage all stakeholders 
to develop deeper, sustainable change in each school. 

The Tennessee Comprehensive Systemwide Plan- 
ning Process (TCSPP) provides the infrastructure for 
sharing and coordinating resources. All 136 districts 
submit their plans to be approved by a cadre of SEA 
staff. In each plan, local personnel develop priorities 
for improving schools using a template and share the 
process by which they arrived at their priorities. 

Kentucky created a Comprehensive Improvement 
Planning system that streamlined district funding 
requests. While the system addresses compliance 
requirements, the planning document is more about 
strategies that the district will use to address its docu- 
mented needs. Kentucky provides annual performance 
reports to help guide the district and school planning 
process. The Kentucky website provides multiple tools 
such as sample plans, the role of a school council, etc. 
Kentucky’s District Improvement Planning Roles and 
Responsibilities training document begins by noting 
that the process is designed to include stakeholders 
(such as parents, staff, and administrators) in creating 
a plan that promotes and supports school improvement 


efforts. Critical steps for improvement planning are 
outlined in the document. 

12.2 Providing technical assistance to improve 
professional practice 

The performance levels for this indicator range from 
the SEA not having policies for how it provides 
technical assistance to districts/schools about how 
to improve professional practice to the SEA having 
an ongoing process for evaluating and improving its 
policies for proving technical assistance to improve 
professional practice. 

In order to continuously assess the effectiveness of its 
system of recognition, accountability, and support as 
well as the efficacy of the strategies it recommends for 
schools, Oklahoma holds What Works in Oklahoma 
Schools institutes twice a year for all schools need- 
ing serious improvement. The presentations highlight 
strategies employed by Oklahoma schools that have 
demonstrated effectiveness, including strategies related 
to: instruction; school culture, leadership, motivation, 
and discipline; professional learning communities; and 
collection, analysis, and use of data (Corbett, 2011). 

12.3 Building parent involvement into school 
improvement 

The performance levels for this indicator range from 
the state having no current written plan to provide 
training on how to include parents in the school 
improvement process to the state having evidence that 
the districts/schools have implemented the strategies 
promoted in training, and the state refines the plan 
based on systematic evaluation of the effects of the 
training. 

A principal interviewed in Kentucky noted that paren- 
tal involvement and special education were two areas 
in which the administrators received especially ben- 
eficial assistance. They noted that the state has helped 
them quite a bit with parental involvement. “In the 
development of our district improvement plan, they 
were very picky about our parent involvement activity, 
and they have provided us support to structure commu- 
nication with parents about holding high expectations 
for all students.” 

Kentucky’s District Improvement Planning Roles and 
Responsibilities document begins by noting that the 
process is designed to include all stakeholders (par- 
ents, staff, and administrators) in creating a plan that 
promotes and supports school improvement efforts. 
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12.4 Evaluating external partners 

The performance levels for this indicator range from 
the SEA having no policies to conduct a rigorous 
review of external providers to support district/school 
improvement to the SEA has an ongoing process for 
evaluating and improving the effectiveness of its poli- 
cies for reviewing external providers. 

Corbett (2011) notes that Elawaii designed a restructur- 
ing framework to assist the state’s growing number 
of persistently low-achieving schools. The Elawaii 
Department of Education (EIIDOE) recognized the 
need for additional (and external) supports and exper- 
tise to turn around the schools and issued an RFP for 
vendors. Ten vendors responded, and a five-person 
committee evaluated the applications. The state 
selected three Lead Turnaround Partner-type organiza- 
tions to manage the restructuring effort in 20 schools. 
In subsequent years, RFP responses were vetted by a 
committee of 25 professionals. The RFP and procure- 
ment process is run by EIIDOE ’s Special Programs 
management section. This division significantly 
reduces the amount of work and time needed to select 
and contract with the external providers. The state 
monitors the partnerships regularly; one of the original 
providers was removed due to insufficient gains in 
schools. 

12.5 Implementing content standards that prepare 
students to take credit-bearing courses at post- 
secondary institutions and for a career 

The performance levels for this indicator range from 
the SEA having no policies for assisting districts/ 
schools in implementing content standards for post 
secondary experiences to the SEA having an ongoing 
process for evaluating and improving the effective- 
ness of its policies for assisting local educators to 
implementing content standards for helping to prepare 
students for post-secondary institutions. 

In its 2012 flexibility waiver request, Tennessee noted 
that it committed to raise standards and expectations 
for all students by adopting the Common Core State 
Standards (CCSS). Tennessee is using this adoption 
with aligned assessments and training as the stimulus 
to improve instruction with emphasis on rigorous con- 
tent and critical thinking thereby subsequently improv- 
ing student achievement. 

The state’s approach in implementing these new con- 
tent standards is to develop a more effective model of 
professional development. It established a Leadership 


Cabinet (that will work in partnership with the SEA’s 
division of Curriculum and Instruction) that will be 
comprised of 1 0 district leaders and will oversee the 
design of teacher trainings and communication across 
the state over the next 3 years. They will work with a 
body of Master Teachers, comprised of three teachers 
per grade level for each Field Service Center region. 
Together, the Leadership Cabinet and Master Teachers 
will develop and facilitate trainings for school-level 
coaches on CCSS. In addition, Tennessee will provide 
video and online modules specific to each grade level 
and content area for review by teachers and parents 
(Tennessee, 2012). 

13. Dissemination of knowledge 

13.1 Disseminating knowledge and/or research 
based practices 

The performance levels for this indicator range from 
not having a process at the state level for making prod- 
ucts available to help districts and schools with school 
improvement to evidence that the SEA evaluates the 
use by districts/schools of the knowledge disseminated 
by the state to improve student achievement. 

In Alabama, several newsletters are disseminated 
on school improvement. Regional in-service centers 
based in universities publish newsletters. Regional 
school improvement coaches develop and disseminate 
newsletters to districts within their regions, provid- 
ing notification of current training opportunities and 
plans for future training. Professional development is 
a key area of state support, delivered via intermediate 
centers, peer mentors, online training, and state confer- 
ences. The local administrators describe the profes- 
sional development as being ongoing, of high quality, 
and tailored to their specific needs. 

The Washington SEA developed a statewide math and 
science instructional coaching program, including a 
coach development institute, coaching seminars, and 
coaching activities within schools. Evaluation find- 
ings showed that the role, support, and responsiveness 
of the SEA have consistently received high marks; 

90% of staff indicated that professional development 
provided by the state resulted in changes in their class- 
room instruction. In addition, Washington produces 
resource guides to assist the external facilitators and 
local staff. The Nine Characteristics of High Perform- 
ing Schools (2007) is a research-based framework for 
school improvement in Washington. Each of the stud- 
ies that serve as the basis for this guide was analyzed 
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to determine which characteristics were found most 
often among high-performing schools. 

Kentucky’s plans for the statewide longitudinal data 
system includes a Knowledge Management Portal 
that will provide a wealth of targeted instructional 
resources, including standards-based units of study, 
lesson plans, curriculum maps, assessments, and other 
educational resources. The portal will offer collab- 
orative workspace that teachers can use to share best 
practices, develop test items, and expand their profes- 
sional skills. 

13.2 Producing products and resources to help 
districts and schools improve 
The performance levels for this indicator range from 
the state having no products to help districts and 
schools with school improvement to having evidence 
that the districts and schools are using the available 
products and find them to be beneficial. To receive the 
highest rating, the state must also evaluate and modify 
those products. The evaluation also guides the state in 
its search for additional relevant resources. 

In the State of Washington, a key resource for schools 
and districts is the State Department of Education web- 
site where they provide data, professional development 
materials, grade-level expectations, information from 
the Nine Characteristics of Effective Schools, and the 
School System Improvement Resource Guide, which 
provides a systematic framework for those embark- 
ing upon school and district improvement. Educators 
interviewed singled out curriculum, increasing paren- 
tal involvement, and assistance in serving ELLs as 
areas in which the state has provided especially helpful 
materials and training. 

14. Monitoring, program audits, and diagnostic site 

reviews 

14.1 Conducting state monitoring, program audits, 
and diagnostic site reviews 
The performance levels for this indicator range from 
the state having no system in place to conduct program 
audits in schools and districts to evidence that the SEA 
has an ongoing process for evaluating the effective- 
ness of its policies for monitoring, conducting program 
audits, and providing diagnostic site reviews. 

Tennessee has relied on its 2007 publication, What is 
a Good School Appraisal Guide and Rubric, as the 
focus of its work with districts on curriculum, instruc- 
tion, organization, and assessments (both formative 
and summative). This document provides a consistent 


message to local staff that “. . .teams of educators visit 
schools across Tennessee to determine if teachers are 
really teaching and if all students are really learning to 
the best of their potential.” The site visits employ a set 
of criteria for effective schools and a connected set of 
standards and measurement statements with matched 
rubrics. The site visits result in an individual school 
profile of strength and areas of planning needs. 

Washington State’s School and District Improvement 
Division has implemented a School Performance 
Review (ED Audit) system. School performance 
reviews are conducted with all schools that have vol- 
unteered to receive School Improvement Assistance 
Program services. ED Audit teams, consisting of six 
to ten professionals, spend four to five days on-site 
looking at curriculum and instruction through the 
lens of the Nine Characteristics of High Performing 
Schools and alignment with the research on high- 
performing districts. Data sources that support the 
School Performance Review include classroom visits 
and observations, interviews, focus groups, and other 
documentation. The School Performance Review final 
report is one component of a data portfolio that staffs 
consider and discuss for data-driven decisions and the 
revision of the school improvement plan. Within two 
weeks of the visit, the School Performance Review 
team leader presents the report to the full staff; the 
school is responsible for sharing the report with their 
community for input. The final report becomes one 
part of the school improvement process. 

Montana staff adapted the Kentucky Department of 
Education’s model to develop its statewide system 
of support. Priority schools all undergo a scholastic 
review to assess three areas: academic performance, 
environment, and efficiency. The purpose of the 
review is to analyze the strengths and limitations of the 
school’s instructional and organizational effectiveness. 
The findings are used to make specific recommenda- 
tions to improve teaching and learning. 

14.2 Documenting the status of districts/schools 
The performance levels for this indicator range from 
districts/schools needing substantial improvement 
in student achievement and/or graduation rates have 
been identified, but it is not clear for how long or the 
rationale for placement to the SEA having an ongoing 
process for evaluating the effectiveness of its policies 
for identifying districts/school and reporting results of 
interventions and supports. 
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Florida, in its 2012 flexibility waiver request, docu- 
ments its process for displaying the number of schools 
needing assistance as well the evaluation results 
illustrating the effectiveness of its intervention policies 
(Florida, 2012). 

14.3 Monitoring the progress of individual 
districts/schools 

The performance levels for this indicator range from 
the state having no process in place to formatively 
evaluate the progress of districts/schools on ESEA 
sanctions lists to using formative evaluation data, 
monitoring the progress of districts/schools, suggest- 
ing modifications in local staff practices, and providing 
evidence that such modifications have been imple- 
mented by districts/schools. 

In Delaware, the application, evaluation, and monitor- 
ing systems provide the infrastructure to support the 
SRAS. Delaware has revised the District and School 
Improvement Plans (now District and School Success 
Plans) so that they articulate the complete strategic 
plan for the agency — one plan that encompasses all 
students. 

The Ohio Department of Education assigns State 
Diagnostic Teams (SDT) to review districts in need 
of the most support. They conduct thorough examina- 
tions of districts and their schools including, among 
other things, walk-through observations and focus 
groups, and then report their findings. School Support 
Team (SST) members make an effort to attend review 
meetings between local staff and the SDTs, serve as a 
liaison between the local staff and the SDT, and help 
the districts design action plans that address the weak- 
nesses identified in the SDT’s reports. With regard to 
monitoring progress, the SSTs measure impact after 
training and follow up with the districts to see what 
changes in practice have taken place and the effects of 
those changes. 

Oregon’s SRAS facilitators meet regularly with the 
school improvement staff and monitor evidence of 
implementation to determine where additional assis- 
tance may be needed. 


Part D: Outcomes for Schools Served by the 
System of Recognition, Accountability, and 
Support 

15. Outcomes/results 

15.1 Establishing student achievement 
performance targets 

The performance levels for this indicator range from 
the SEA not establishing performance targets nor cri- 
teria on how to use the assessment results to the SEA 
having an ongoing process for evaluating the effective- 
ness of its policies relative to setting high performance 
targets and criteria to identify the progress of all its 
schools. 

In its 2012 flexibility waiver request, Tennessee stated 
that it had set rigorous proficiency goals to measure its 
progress as a state and used those goals to set district 
performance targets. “These goals are our line in the 
sand. They represent significant, steady growth in 
student achievement that would change Tennessee’s 
educational trajectory as a state. We have proposed 
increasing our reading and math proficiency rates by 
around 20% over a five year arc.” Tennessee is calling 
upon each district and school to grow from its current 
starting point, continuously improving each year (Ten- 
nessee, 2012). 

15.2 Addressing subgroup achievement gaps 
The cells in this indicator range from the SEA not 
having a process for districts/schools to use their 
achievement gap data as part of their improvement 
strategies to the SEA being able to evaluate the effec- 
tiveness of its policies and procedures in assisting 
districts and schools to implement strategies to reduce 
subgroup achievement gaps. 

Kentucky has a statewide network of local District 
Assessment Coordinators as well as a separate group 
of District Achievement Gap Coordinators (DAGCs) — 
five highly effective educators assigned to work in 
selected regions of the state to eliminate achievement 
gaps. The DAGCs collaboratively develop individual 
work plans for each assigned district based on the 
results of the district’s Scholastic Audit. In addi- 
tion, these DAGCs serve as members of five-person 
teams that assist districts identified as needing seri- 
ous assistance and who request Voluntary Partnership 
Assistance Teams. Receiving help from these teams is 
one of the three options the state has made available 
to districts that have failed to meet adequate yearly 
progress for four years. 
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The Office of Achievement Gap Elimination (AGE) 
was established by the Tennessee legislature in 2007. 
AGE consists of an urban specialist and several con- 
sultants, all employed by the SEA. AGE consultants 
provide assistance to Tennessee’s High Priority School 
Improvement schools. The focus of this office is to 
close achievement gaps for the following subgroups: 
students with disabilities; economically disadvantaged; 
limited English proficient; African American males; 
other subgroups as appropriate. 

In addition, AGE consultants are assigned to Title I 
schools in their first year of improvement to provide 
them with direct technical assistance on the develop- 
ment and implementation of their improvement plans. 
To begin this process, AGE consultants partner with 
trained distinguished educators to conduct thorough 
audits of the schools using the practitioner guide, What 
is a Good School? From the results of these audits, 
the AGE consultants guide the schools through the 
development of their Title I school improvement plans, 
including the targeting of resources available through 
the Title I school improvement grant funds. 

15.3 Establishing student attendance performance 
targets 

The performance levels for this indicator range from 
the SEA not having established attendance perfor- 
mance targets for districts/schools to the SEA having 
an ongoing process for evaluating the effectiveness of 
its policies for setting attendance targets. 

In its 2012 flexibility waiver request, Florida lists stu- 
dent attendance as a key factor to be addressed when 
the regional Differentiated Accountability instructional 
coaching staff makes site visits to schools. 

15.4 Establishing graduation rate performance 
targets 

The performance levels for this indicator range from 
the SEA not having established graduation perfor- 
mance targets to the SEA having an ongoing process 
for evaluating the effectiveness of its policies for 
setting high graduation targets and showing evidence 
that it reliably and validly measures student graduation 
results. 

Tennessee notes in its 2012 flexibility waiver that it 
has set an AMO (Annual Measurable Objective) for 
graduation for all students (90% by 2014-15) as well 
as AMOs for sub-groups. Their overall goal is for all 
subgroups of students to reach a graduation rate of 


90% over time. Since presently different sub-groups of 
students are graduating at different rates, the SEA has 
set differentiated targets through 2018-19. 

In its 2012 flexibility waiver request, Florida noted that 
high schools with graduation rates calculated to be the 
lowest in the state or subgroup graduation rates that 
are significantly lower than the overall school, district, 
or state rate will be reported to their regional centers, 
and the school and district will be required to include 
specific strategies in their district/school improvement 
plans to increase the graduation rates. 


65 


Hanes, Kerins, Perlman, & Ross 


References 

Corbett, J. (2010, October 30). The Virginia story’. Lincoln, 
IL: Center on Innovation & Improvement. 

Corbett, J. (2011). Lead turnaround partners: How the 
emerging marketplace of lead turnaround partners is 
changing school improvement. Lincoln, IL: Center on 
Innovation & Improvement. 

Corbett, J. (2011, March). The Oklahoma story. Lincoln, 

IL: Center on Innovation & Improvement. 

Corbett, J. (2011, May). The Montana story. Lincoln, IL: 
Center on Innovation & Improvement. 

Florida State Flexibilty Waiver. (2012). Retrieved from 
http://www2.ed.gov/policy/eseaflex/approved-requests/ 
fl.pdfp. 116 

Kerins. T., Perlman, C., & Redding, S. (2009). Coherence 
in statewide systems of support. Lincoln, IL: Center on 
Innovation & Improvement. 

Lane, B. (2010). Transforming a statewide system of sup- 
port: The Idaho story. Lincoln, IL: Center on Innova- 
tion & Improvement. 

Pope, J. (2012, May 5). AP surges as tool for schools rais- 
ing standards. Associate Press. 

Redding, S., & Walberg, H. J. (Eds.) (2007). Handbook on 
statewide systems of support. Charlotte, NC & Lincoln, 
IL: Information Age Publishing & Center on Innovation 
& Improvement. 

Tennessee State Flexibility Waiver. (2012). Retrieved from 
http://www2.ed.gov/policy/eseaflex/approved-requests/ 
tn.pdfp 19 


66 


Illustrating the Scoring System 

Steven Ross, Carole Perlman, C. Thomas Kerins, Susan Hanes 


Explaining the Rubric 


SRAS Rubric Exemplars 

(Mostly from states’ open-ended questionnaire responses) 


Part A: Design and Assessment of the SRAS 


1 

SRAS design and differentiation 

1.1 

Designing and 
organizing an 
SEA System of 
Recognition, 
Accountability, and 
Support (SRAS) 

The state education agency (SEA) has been working to advance an agenda that 
it outlined in a key planning document that was released nine months ago, titled 
Forward Thinking. This plan articulates a mission, goals, and timelines for SEA 
staff to provide better service to the field in support of enhanced student results. 
To disseminate this message, the Commissioner of Education visited 85 of 178 
districts. In these visits, he worked with superintendents one-on-one to explain 
how this change would help them. 

The reviewers rated the state as operating at Level II on this indicator. Although 
the planning document was viewed as a good beginning toward creating a fully 
documented SRAS system, still lacking is documentation (to achieve Level III) 
of a document, such as an operations manual, that more explicitly describes the 
roles and responsibilities of offices within and external to the SEA as well as 
some evidence that SEA staff are implementing actions based on this new plan. 

Exemplary Evidence of High Performance 
0 SRAS operations manual 
0 Organizational chart for SRAS functions 
0 Role descriptions of key SRAS staff 
0 Evidence of new SEA actions based on the SRAS plan 

1.2 

Engaging stakeholders 
to solicit input on 
the development and 
improvement of the 
SRAS 

In order to be consistent in its proposals, the state established both a process and 
a list of important stakeholders to include in the development of both proposals 
as well as state program efforts, such as its SRAS. The process includes mail- 
ings, establishment of a website, regional meetings and a host of presentations 
at media gatherings. The personnel include K-12 educators, IHE representatives, 
Civil Rights groups, representatives from advocacy organizations for students 
with disabilities and English Language Learners. As a matter of policy, the state 
summarizes the major supporting statements as well as objections. The state 
then acknowledges what changes it has made because of comments by stake- 
holders as well as which ones it has not adopted and for what reasons. 

The reviewers rated the state as operating at Level IV on this indicator. The state 
has taken involvement of stakeholders seriously and has evidence to prove it. 

Exemplary Evidence of High Performance 
0 List of organizations included in the dissemination 
0 List of presentations and the audiences 

0 Document noting what changes have been made as a result of input by 
stakeholders 
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1 

SRAS design and differentiation 

1.3 

Managing the SRAS 

To ensure that resources are leveraged in a way that aligns with the new mis- 
sion of increased service to the field, the Commissioner authorized a study 
(conducted by an external organization) that has identified how SEA resources 
can better align with priorities. Based on that study, SEA staff devised a written 
plan for coordinating services. Last fall, SEA staff, representing several federal 
and state programs, began using the plan to identify high-priority service areas 
(e.g., early literacy) identified as most in need of higher efficiency in resource 
allocation. 

The reviewers rated the state as operating at Level III based on its development 
and implementation of a written plan for coordinating resources. To achieve 
Level IV, a more systematic and full implementation of the plan, with evidence 
of positive outcomes in achieving greater efficiency, would be needed. 

Exemplary Evidence of High Performance 
0 A written plan for oversight and coordination of resources 

0 Meeting agendas and other records of communications (with districts/ 
schools) 

0 District/school data demonstrating implementation of state plan 
0 Survey data from districts/schools regarding use of plan, outcomes, etc. 

0 Statewide or case specific data demonstrating reduction of duplicated 
services, greater efficiency in services, cost savings, etc. 

1.4 

Staffing the SRAS 

The SEA has established criteria for the selection and hiring of personnel for 
its SRAS. It has carefully matched the abilities of staff with the responsibili- 
ties defined in its operations manual. In addition, training programs have been 
developed and implemented to assure local education personnel that SRAS staff 
has the knowledge and skills to provide technical assistance. 

The reviewers rated the state as operating at Level III because it has not yet 
established an evaluation process to see if local personnel who are receiv- 
ing help from SRAS personnel have an ongoing opportunity to evaluate these 
individuals. As local needs change, the question is whether the SRAS staff will 
receive updated training and materials. 

Exemplary Evidence of High Performance 
0 List of criteria for SRAS staff selection 
0 Operations manual with specific job responsibilities 

0 Ongoing survey data from school personnel evaluating the quality of the 
technical assistance they receive from SRAS staff 
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1 

SRAS design and differentiation 

1.5 

Integrating the SRAS 
within the SEA 

The state has a written plan demonstrating how various programs within the 
SEA can work together to more efficiently provide services to districts. Because 
of a recent reduction in the number of employees, it has become imperative 
that the SRAS provide services more efficiently. To do this, staff from vari- 
ous departments were brought together to write a plan for joint sharing of data, 
monitoring responsibilities, and the development of training programs. 

The reviewers rated the state as operating at Level II for this indicator since 
the results of the joint planning had not yet been implemented. There were only 
examples of how the cooperative efforts would work. 

Exemplary Evidence of High Performance 

0 The written plan showing how cooperative efforts would take place and a 
schedule documenting when this would begin 

0 A plan documenting how the state would notify local personnel of the 
proposed changes 

0 An evaluation plan to document whether the changes are working 
internally and whether local personnel are receptive to the changes 

1.6 

Differentiating support 
to districts and schools 

The state has developed a set of criteria that prioritizes districts/schools accord- 
ing to need. Student achievement is given the highest weighting, but other fac- 
tors include graduation rates, student mobility, and teacher qualifications. Rather 
than developing a single, quantitative index, the state uses these data to catego- 
rize schools and districts into four “need” categories (critical, high, moderate, 
and low). Based on an analysis of need and services already rendered, support 
is tailored to meet the unique needs of each high-priority school based on the 
school’s comprehensive needs assessment and school improvement plan. 

The reviewers rated the state as operating at Level IV. Evidence showing that 
support was being implemented differentially according to school and district 
needs and that results are evaluated was judged to meet the criteria for the high- 
est performance level on this indicator. 

Exemplary Evidence of High Performance 
0 A written plan documenting criteria for differentiating support 

0 Data showing differentiation of districts/schools (e.g., tiered listings of 
status by needs) 

0 Data describing the provision of differentiated services to districts/ 
schools in relation to specific criteria (e.g., priority level, percentage of 
disadvantaged students, etc.) 

0 Evaluation reports documenting the changes that have occurred as a 
result of the support provided by the SRAS 
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1 

SRAS design and differentiation 

1.7 

Improvement planning 
and implementation 
process for districts 
and schools 

The state has developed a document, Improvement Planning for Raising 
Achievement for use by districts/schools. The document essentially provides a 
checklist of key areas to target, such as conducting a needs assessment, using 
data to drive improvement, recruiting/hiring highly effective teachers, select- 
ing evidence-based programs, etc. The document is posted on the SEA’s SRAS 
website, and all districts and schools are encouraged to use it. 

The reviewers rated the state as operating at Level II. Although the state has a 
clearly defined and documented planning process for districts/schools, it still 
needs to develop and begin an implementation program (including training and 
materials) to reach Level III and show evidence of schools’ and districts’ partici- 
pation to achieve Level IV. 

Exemplary Evidence of High Performance 
0 Documented district and school improvement planning processes 
0 Training materials/tools for developing improvement plans 
0 Agendas, handouts, and participation lists for training sessions 
0 Survey data from districts/schools supporting usage of the process 

0 District/school data demonstrating systematic improvement planning (e.g., 
a sampling of actual district and school improvement plans, identified 
reform interventions, etc., which could be accomplished more easily if the 
state had an electronic planning system such as Indistar®) 

0 Plan for monitoring districts’ and schools’ implementation of their 
improvement plans 

1.8 

Providing 

differentiated services 
and resources to 
support district and 
school improvement 

The state has a plan to provide different levels of assistance and resources to 
support district and school improvement. A formula has been developed to docu- 
ment the depth of services necessary to help the most needy districts/schools. 
The SRAS is organized to focus on these priority districts/schools while still 
providing important resources to districts/schools that have particular needs, 
such as improving the performance of Hispanic students in middle school math. 

The reviewers rated the state as operating at Level III because it did not have an 
evaluation plan to document that it was making the right decisions in providing 
certain districts with the most comprehensive services. 

Exemplary Evidence of High Performance 

0 List of districts/schools receiving the most comprehensive services as well 
as documentation of what resources they have receive for what period of 
time 

0 Survey results from all the districts/schools receiving differentiated 
services to ascertain if they believe the right match has been made 
between services and their needs (with specific examples given) 
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1 

SRAS design and differentiation 

1.9 

Intervening in districts 
and schools that 
repeatedly do not meet 
targets for student 
achievement and 
graduation rates 

SEA has a written plan and authority for intervening with the lowest-performing 
schools as well as high schools with low graduation rates. Two critical elements 
of the plan are the (a) systematic tracking of school needs for SRAS and ser- 
vices received, and (b) use of school improvement consultants to help schools 
conduct needs assessments and implement appropriate “turnaround” interven- 
tions. In addition, the SEA has brought local superintendents and local school 
boards to the forefront of accountability by including them in discussions related 
to interventions in specific schools. 

The reviewers rated the state as operating at Level III based on its creation and 
implementation of a systematic plan. The involvement of the superintendents 
was viewed as a positive step toward putting in place a formal corrective action 
process and an evaluation (needed for Level IV). 

Exemplary Evidence of High Performance 

0 A written policy, plan, and specific criteria for intervening in low-performing 
districts/schools 

0 Description of services provided to identified sites by school improvement 
teams 

0 Agendas/descriptions of meetings with “turnaround” district/school leaders 

0 Communications/records regarding corrective actions planned with 
targeted sites 

0 Documentation of actions/interventions taken with each district/school and 
subsequent results 


2 

Supports and interventions for all students and subgroups 

2.1 

Helping schools 
and districts better 
serve students with 
disabilities 

To assist schools and districts in raising the achievement of students with 
disabilities, SEA uses an assessment tool, the LRE Assessment, in its SRAS 
to increase regional and LEA capacity to serve special education students. It 
provides additional help on an as-needed basis via site visits, consultants, and 
workshops. 

The reviewers rated the state as still operating at Level I due to the absence of a 
formal, documented plan for assisting districts/schools in serving special needs 
students. Once a plan is developed, the rating could rise to Level III due to the 
ongoing implementation of the service activities described. 

Exemplary Evidence of High Performance 

0 A written plan for assisting districts/schools failing to meet targets for 
students with disabilities 

0 Description of actual services provided in accord with the written plan 

0 Communications/records showing efforts to provide assistance to districts/ 
schools in serving students with disabilities 

0 Survey data from districts/schools regarding usefulness of the state 
assistance provided and indicating one or more specific examples 
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2 

Supports and interventions for all students and subgroups 

2.2 

Coordinating services 
for students with 
disabilities across 
SEA departments and 
programs to maximize 
service and reduce 
duplication 

The SEA has held several meetings to engage the various departments in 
discussing ways of coordinating work efforts to support the needs of districts. 
Special education personnel are members of the School Improvement Round- 
table, which coordinates the inter-departmental school improvement efforts, and 
helped to write the formal state plan for integrating special education services 
to districts. As part of the plan, special education specialists participate in all 
LEA Resource Team District visitations. The special education personnel and 
the Learning Resources Services in the regions collaborate with regional support 
teams to provide special education intervention and improvement services to 
schools. 

The special education personnel work with the school improvement person- 
nel by coordinating special education monitoring with System Performance 
Reviews and share monitoring data with the Assessment of Performance on 
School Standards analysis teams. 

The reviewers rated the state as approaching Level IV, but still operating at 
Level III. Although there is a documented plan and many coordination activities 
are taking place, the review team did not see evidence indicating how successful 
this coordination has been. No criteria to judge success have been developed. 

Exemplary Evidence of High Performance 

0 A written plan for integrating district/school assistance policies for serving 
students with disabilities 

0 Description of services provided by special education and other SEA 
personnel in accord with the written plan 

0 Data collection/monitoring tools used in working with schools and districts 

0 Reports, organizational charts, or program descriptions showing increased 
coordination of services for students with disabilities 

0 Survey data from districts/schools corroborating increased quality/ 

efficiency of state services and reduced redundancy in data collection and 
reporting, and indicating one or more specific examples 
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2 

Supports and interventions for all students and subgroups 

2.3 

Helping schools and 
districts better serve 
English language 
learners 

In accord with the documented plan, the state SRAS and the Office of Language 
Acquisition (OLA) operate within the Accountability and Targeted Assistance 
(ATA) cluster. This year, OLA staff actively participated in SRAS initiatives in 
all seven identified districts. In addition, ELL education facilitators/specialists 
participate in all LEA Resource Team District visitations. The third major part of 
the plan implemented last year was usage of an assessment tool by the SRAS to 
increase regional and LEA capacity to serve English language learners. 

The reviewers placed the state at Level IV. Implementation of the primary 
components of the overall plan for this indicator was judged to be reasonably 
complete and an evaluation system had begun. 

Exemplary Evidence of High Performance 

0 A written plan for assisting districts/schools failing to meet targets for 
English language learners 

0 Description of services provided by ELL education facilitators or other SEA 
personnel in accord with the written plan 

0 Communications/records showing meetings, site visitations, and other 
activities to help districts/schools meet ELL targets 

0 Data collection/monitoring tools used in working with schools and districts 

0 Survey data from districts/schools regarding the effectiveness of state 
assistance provided, and indicating one or more specific examples 

2.4 

Coordinating services 
for English language 
learners across SEA 
departments and 
programs to maximize 
service and reduce 
duplication 

SEA has worked with the directors of the regional centers to help enact new 
legislation in the state to better serve English language learners. This activity 
is increasing the capacity of regional centers to support improvement efforts in 
small and rural districts. The department partners with the regional centers to 
provide enhanced regional trainings in identified areas of need. As an example, 
one pilot partnership focuses on the Native American student population. It 
ensures targeted interventions for preschool and kindergarten students. The pilot 
brings together services from English language acquisition, special education, 
and early childhood units. SEA is developing a plan to formalize these and other 
services. 

The reviewers viewed the state as active and achieving some success in coordi- 
nating ELL services. However, it assigned a rating of Level I on this indicator 
due to the absence of a formal plan (which is presently in preparation). Without 
such a plan, the desired coordinating activities would not be clearly defined, 
thereby decreasing potential for accountability and sustainability. 

Exemplary Evidence of High Performance 

0 A written plan for integrating district/school assistance policies to serve 
English language learners 

0 Description of services provided by ELL staff or other SEA personnel in 
accord with the written plan 

0 Reports, organizational charts, or program descriptions showing increased 
coordination of services for ELL students 

0 Survey data from districts/schools corroborating increased quality and 
efficiency of state services and reduced redundancy in data collection and 
reporting, with one or more specific examples 
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3 

SRAS evaluation design 

3.1 

Documenting district/ 
school activities 
provided through 
SRAS 

The SEA measures each participating district’s and school’s progress against 
18 identified performance targets. A data “dashboard” is used to periodically 
check progress on each of the goals and can be seen on the SEA’s SRAS web- 
site. There are data points that are associated with each of the SRAS goals. An 
example of an indicator would be “number of schools in the district reducing 
the achievement gap among subgroups (see Indicator 15.2). Increasing the high 
school graduation rate of Native Americans is another example of a performance 
target. The displays on the SEA website show the status of each individual 
school in detail as well as supportive activities provided by the SRAS. The 
SEA is also currently initiating an electronic school improvement management 
system that will document progress on the performance targets at a glance and 
will show the types of interventions provided for each school. These data will 
be used to evaluate the patterns of effectiveness of certain types of supportive 
activities provided for school staff by the SEA’s Division of School and Leader 
Quality as well as the school improvement teams it supports. 

The reviewers classified the state as operating at Level IV on this indicator. 
Based on the evidence, the state clearly was judged as having a fully operational 
system for documenting work with districts and schools. 

Exemplary Evidence of High Performance 

0 A written plan describing the process for documenting SRAS work with 
schools and districts 

0 A report or formal description of the actual SRAS work performed 

0 Data showing school and district outcomes in relation to SRAS services 
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3 

SRAS evaluation design 

3.2 

Evaluating the SRAS 

The SEA has a documented plan for surveying all districts and schools served 
by SRAS. To date, it has administered an “Annual Stakeholder Survey” to 
attendees at all SRAS state and regional meetings. In addition, researchers at a 
local university recently completed a mixed methods study of perceptions about 
SRAS interventions by three urban and three rural districts that received ser- 
vices over the last two years. 

The reviewers rated the state as achieving Level II on this indicator. No crite- 
ria or standards to judge success have been developed to be used to judge any 
collected data. Although a formal plan has been developed, the evaluation work 
thus far has been informal (relying on “convenience samples” of those attend- 
ing meetings) rather than systematically implementing the evaluation plan. The 
latter activity would elevate the performance to Level III, whereas actual usage 
of the data to improve services would be needed to achieve Level IV. 

Exemplary Evidence of High Performance 

0 A written plan for collecting data on districts’ and schools’ perceptions of 
SRAS services 

0 Copies of the survey(s), interviews, or other instruments administered 

0 A description of data collection activities and schedules 

0 A list of the criteria to be used to judge the success of the SRAS 

0 A report on the findings from any studies done to judge what difference 
the SRAS has made with regard to effecting increases in districts/schools 
making performance targets 

0 Documents, reports, or other evidence that the perception data has been 
used for improving the SRAS 
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3 

SRAS evaluation design 

3.3 

Evaluating the SEA’s 
assessment program 

Given the use of revised content standards, an SEA intends to hire external 
consultants to measure the alignment of the state tests with these new content 
standards. The consultants will also be asked to evaluate the extended content 
standards developed for a percentage of the students with disabilities subgroup 
to see if they align with these standards. They will look at technical measures 
including reliability. Finally, these consultants will evaluate the effectiveness of 
the training prior to the test as well as the materials distributed to schools and 
parents to explain the results and how they can be used for student and school 
improvement. 

The reviewers rated this SEA as operating at Level I on this indicator. Although 
the state is planning to conduct a rigorous and comprehensive evaluation using 
an external vendor, they are presently still in the “building” phase with regard to 
both the evaluation plan and its implementation. 

Exemplary Evidence of High Performance 

0 A written evaluation plan to determine the effectiveness of SEA 
assessment program 

0 Copies of the data collection instruments and analysis plans 

0 A description of data collection activities, schedules, and the criteria that 
will be used to make judgments 

0 A report on the findings from the study 

0 Documents, reports, or other evidence that the evaluation results have 
been used for improving SRAS (new programs or policies, a SRAS 
improvement plan, etc.) 
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Part B: Resources and Services Provided to Districts and Schools 


4 

District and school staff needs 

4.1 

Enhancing the supply 
of teachers and 
leadership personnel 
skilled in school 
improvement strategies 

According to the state’s plan, the SEA regularly delivers professional devel- 
opment to turnaround school administrators and superintendents in districts/ 
schools it has identified as high priority. For increasing the supply of skilled 
teachers and principals, the SEA plan also identifies strategies for helping insti- 
tutions of higher education (IFlEs) to prepare teachers and administrators more 
effectively. In addition, high-need schools are identified for the Teacher Quality 
and Equity Action Plan for Title IIA. These schools are offered technical assis- 
tance and funding to increase the percentage of highly effective teachers and 
decrease the percentage of inexperienced teachers. The state disseminates infor- 
mation on incentives offered by districts via its website and other publications. 
Last year, the state increased its supply of skilled teachers and administrators by 
15% and 20%, respectively. 

The reviewers rated the state as operating at Level IV on this indicator. Spe- 
cifically, the state has a documented plan, appears to be implementing it using 
multiple strategies (incentives, training, and information), and has implemented 
an evaluation to document the increase in its supply of highly effective teachers 
and administrators over the previous year. 

Exemplary Evidence of High Performance 

0 A written plan for increasing the supply of teachers and administrators 
skilled in school improvement 

0 Documents/artifacts reflecting the strategies used 

0 Description of recruitment or outreach activities provided by SEA 
personnel in accord with the written plan 

0 District/school data showing increased supplies of skilled teachers or 
administrators (e.g., % working in low-performing schools compared to 
prior years) 

4.2 

Providing incentives 
and strategies for 
addressing a more 
equitable distribution 
of well qualified and 
effective teachers 
within and across 
districts 

The state annually collects and reports the percentage of teachers by years of 
experience for both high- and low-poverty districts. It also annually reports the 
percentage of core academic subjects taught by teachers considered not highly 
effective according to the SEA’s teacher evaluation system. 

The reviewers rated the state as operating at Level I because there was no plan 
to respond to the findings of its annual data collections. 

Exemplary Evidence of High Performance 

0 A written plan for encouraging highly effective teachers to teach in low- 
performing schools 

0 Criteria for identifying highly effective teachers 

0 Documents/artifacts reflecting the strategies used to encourage a more 
equitable demonstration 

0 Description of recruitment or outreach activities provided by SEA 
personnel in accord with the written plan 

0 Data indicating that more equitable distributions have been achieved 
state-wide (e.g., percentage of highly effective faculty in identified schools 
compared to higher-performing schools; percentage of highly effective 
faculty working in different regions, etc.) 
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4 

District and school staff needs 

4.3 

Recruiting and retain- 
ing well-qualified and 
effective teachers 

This state believes that school improvement cannot occur in many targeted 
districts and schools without a strong induction and mentoring program coupled 
with a retention program that includes financial incentives and opportunities 
for teacher leadership. The state has a written plan that was the foundation for 
legislation supporting a beginning teacher mentoring program, which will be 
implemented next year. Regional centers have begun training programs to assist 
teachers in becoming school leaders without necessarily becoming principals. 

The reviewers rated the state as operating at Level III because they had a writ- 
ten plan and were working concurrently at several levels to implement their 
plan. 

Exemplary Evidence of High Performance 

0 A written plan for enhancing recruitment and retention of highly effective 
teachers 

0 Documents/artifacts reflecting the strategies used 

0 Description of recruitment or outreach activities provided by SEA 
personnel in accord with the written plan 

0 District/school data showing improved recruitment and retention of highly 
effective teachers, including one or more specific examples 

4.4 

Recruiting and 
retaining effective 
district and school 
leadership personnel 

After developing a plan of action, this state worked with teams from all of the 
universities that prepare school leaders to begin building a framework based on 
factors research has indicated are associated with effective school leaders. The 
SEA told the university teams, “We’re going to work with you to analyze this 
information, and then help you restructure your leadership training programs. 
Your task then is to go back and complete the redesign of your courses and your 
programs so that it meets this framework and gets at the critical success factors.” 
The program at IHEs has now been implemented for three years. 

The reviewers rated the state as operating at operating at Level III since there 
was an implemented plan to upgrade administrator training, and the state was 
working with higher education personnel from all pertinent universities. Until an 
evaluation plan has been proposed, the SEA cannot be rated at a IV level. 

Exemplary Evidence of High Performance 

0 An SEA written plan for enhancing recruitment and retention of highly 
effective administrators 

0 Criteria for identifying “highly effective” administrators 

0 Documents/artifacts reflecting the strategies used (e.g., partnerships with 
higher education; new professional development programs) 

0 Description of recruitment or outreach strategies provided by SEA 
personnel for local personnel to utilize 

0 District/school data showing improved recruitment and retention of highly 
effective administrators, including one or more specific examples 
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4 

District and school staff needs 

4.5 

Engaging Institutions 
of Higher Education 
(IHEs) to better 
prepare new teachers 
and leadership 
personnel 

The SEA has met continuously with representatives from IHEs over the last two 
years to develop a joint plan to share information with college professors about 
curriculum standards, aligned state assessments, effective professional practice, 
and a host of issues that should better prepare new teachers and leadership per- 
sonnel. IHE leadership has endorsed the process to share this information 

The reviewers rated the state as operating at Level II because no mechanism 
was established to assure that the professors would actually implement the 
agreement in their classrooms. 

Exemplary Evidence of High Performance 

0 Copy of the actual agreement between the SEA and the IHEs 

0 Survey information from superintendents and principals noting whether 
recent graduates were knowledgeable about the state standards, 
curricula, state assessments and effective professional practice 

0 Survey information from college instructors about their inclusion of 
information on state issues as requested by the SEA 

4.6 

Providing guidelines 
for the evaluation of 
teachers and principals 

The SEA worked with the legislature and the governor’s office to establish the 
principles for the evaluation of teachers and principals. It then met with repre- 
sentatives from teacher unions and the Principals’ Association to solicit their 
advice. Subsequently, the SEA held training programs to explain the guidelines 
and to provide suggestions for valid and reliable data collection. 

The reviewers rated the state as operating at Level II because the training 
seemed insufficient for the actual implementation of an evaluation of teachers 
and principals. 

Exemplary Evidence of High Performance 
0 List of stakeholders involved in the development of the guidelines 

0 Training materials used as well as an evaluation of the training materials 
by stakeholders 
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5 

Funding of improvement efforts 

5.1 

Coordinating state 
and federal funding 
streams and programs 

The SEA has developed a written plan to pool money from several federal and 
state funding sources to support school improvement. The rationale was to avoid 
duplicating expenditures for overlapping programs and interventions. Pooling 
technology funds enabled the SEA to obtain more bandwidth, thereby making 
the site where schools enter their improvement plans run faster and more 
reliably. 

The reviewers rated the state as operating at Level II on this indicator. Con- 
sideration was given to the state’s development of a written plan and its initial 
attempts to integrate funding sources in a particular area. Expansion of the inte- 
gration to include other programs and funding sources and actually beginning to 
implement these coordination efforts would elevate the rating to Level III. 

Exemplary Evidence of High Performance 

0 A written plan to efficiently coordinate school improvement programs with 
different funding sources 

0 Reports or communications reflecting efforts to integrate programs 
0 Reports or program descriptions showing successful integration efforts 

0 Data reflecting savings in costs and resources (staff and materials) 

resulting from increased coordination of programs and improved services 
to schools and districts, including specific examples 

5.2 

Assisting districts in 
assessing their use of 
financial resources 
to fund improvement 
efforts 

The state’s written plan consists of two major areas of support. The first is a 
call-in consulting program by the SEA for helping districts and schools align 
financial resources with priority improvement needs. The second component 
consists of regular visits by SEA business staff to schools repeatedly not making 
achievement and/or graduation performance targets. The business staff works 
with school personnel to ensure that discretionary expenditures align with needs. 
Both of these components are regularly implemented as part of the comprehen- 
sive SRAS plan, including surveys of local superintendents about the effective- 
ness of this approach. 

The reviewers assigned the state the highest rating — Level IV — on this indi- 
cator. The critical accomplishment for reaching this level was implementing 
the district/school visitations by budget staff and an evaluation design which 
includes surveys of local superintendents. 

Exemplary Evidence of High Performance 

0 A written plan for helping schools and districts assess their financial 
resources for school improvement 

0 Documents, reports, Internet postings, or other types of written guidance 
for analyzing budgets 

0 Documents/artifacts describing the supports in place (e.g., call-in service 
or site visits by SEA budget staff) 

0 Records of usage of the support services by schools and districts 

0 Survey data from districts/schools corroborating the availability and value 
of the services, and indicating one or more specific examples 
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6 

Data analysis and use 

6.1 

Providing a 
comprehensive SEA 
data system 

The SEA has formed a “Study Committee” consisting of external and internal 
experts in data systems and user representatives (district and school adminis- 
trators). The committee has completed a plan to significantly expand the cur- 
rent data system which lacks student identifiers consistent with those used by 
districts to track student progress and status. The completed system will provide 
comprehensive, user-friendly data to facilitate assessments of school and district 
progress on multiple indicators. 

The reviewers rated the state as performing at Level II on this indicator. While 
the existing data system meets minimum requirements, the SEA is searching for 
funds to implement an expanded system that should more effectively support 
district/school improvement efforts. 

Exemplary Evidence of High Performance 

0 A description of the present integrated data system, verifying its 
usefulness and efficiency (lack of redundancy) 

0 Sample reports (or printouts) from the proposed data system illustrating 
outputs on a variety of indicators 

0 Survey data from districts/schools corroborating the usefulness of the 
proposed expanded state data system 

6.2 

Using assessment data 

An SRAS committee is charged with completing the yearly review of the effec- 
tiveness of its training program as well as how local personnel are using SEA 
assessment data as an element in its improvement planning. The review by regu- 
lar school personnel as well as teachers of students with disabilities and ELL 
teachers consists of using both quantitative survey data as well as qualitative 
results from interviews and observations to identify successes and problems. 

The reviewers evaluated the state’s activities for this indicator as meeting Level 
III requirements. The state was viewed as implementing its formal process, 
but not yet fully documenting (as required for Level IV) how it uses the assess- 
ment system as a tool in the development of further SRAS resources for helping 
districts/schools. 

Exemplary Evidence of High Performance 

0 A written plan for how local educators could use state assessment results 
to supplement their own assessment findings 

0 Documents/artifacts reflecting the strategies used by the SEA to provide 
training for local educators with different needs for information, especially 
trend data for subgroups of students 

0 Description of resources (training, consulting) provided to schools and 
districts on the uses of the SEA assessment system for improvement 
planning 

0 An evaluation plan with criteria for making judgments about the 
effectiveness of current SEA assessment policies and procedures for 
helping districts and schools understand and use assessment data in their 
improvement planning 
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7 

Support teams and im 

movement consultants 

7.1 

Matching districts/ 
schools with support 
teams and district/ 
school improvement 
consultants 

The SEA has formed five teams of school improvement consultants to work with 
identified schools that are most in need of improvement. The teams presently 
operate on an informal system, whereby the SRAS office identifies schools or 
districts viewed as “high priority” on the basis of test scores, district or princi- 
pal requests, or other data. Approximately three-fourths of the low-performing 
districts/schools are being served by school improvement consultants in some 
capacity. 

The reviewers evaluated the state as operating at Level I on this indicator. 
Although consultants and teams have been serving the majority of the low- 
performing districts/schools, a formal plan for matching them to sites and 
structuring their work is still lacking. Thus, services are mainly determined and 
executed informally across the state. 

Exemplary Evidence of High Performance 

0 A written plan for (a) recruiting/screening improvement consultants and 
support teams, and (b) a process for systematically matching schools and 
districts with these consultants and support teams 

0 Documents/artifacts reflecting the strategies used by the SRAS as it forms 
support teams, matches improvement consultants with districts/schools, 
and prioritizes who receives services first 

0 Data reflecting the numbers/percentages of low-performing priority 
schools and districts are served by improvement consultants/support 
teams 

0 School or district data describing specific improvement strategies being 
used by these improvement consultants/support teams 

0 Survey data from districts/schools corroborating the usefulness of the 
services provided, indicating one or more specific examples 
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7 

Support teams and im 

movement consultants 

7.2 

Training, supervising, 
and evaluating support 
teams and district/ 
school improvement 
consultants 

As required by the state plan, all school improvement consultants must par- 
ticipate in three levels of training/professional development: (a) a one- week 
summer session, (b) a two-day refresher in early fall, and (c) ongoing coaching 
mentoring during the consultant’s first year. The “School Improvement Team 
Academy,” which delivers the training, conducts regular formative evaluations 
of the activities and services, using the data to make refinements as needed. 

The reviewers rated the state as operating at Level IV on this indicator. The 
training process was formally defined, comprehensive, fully implemented, and 
subjected to continuing review, evaluation, and subsequent improvement. 

Exemplary Evidence of High Performance 

0 A written plan to continuously prepare improvement consultants and 
support teams to assist districts/schools 

0 Evaluation reports (conducted by the SRAS) from affected district 
personnel describing their needs for assistance from the improvement 
consultants and support teams. 

0 Data reflecting the number of training sessions held for the improvement 
consultants and number of participants attending each 

0 Training agendas and associated materials (guidebooks, PowerPoints, 
handouts, etc.) 

0 Survey data from trainees corroborating the quality of training services, 
and indicating one or more specific examples of the usefulness of the 
targeted training 


8 

External partners and providers 

8.1 

Managing and 
coordinating 
organizational partners 

Based on a written plan, collaborations have been formed with IHEs, Area 
Education Agency staff, other staff within the SEA, and external organizations 
(e.g., School Administrators Association, and State Association for School 
Boards). The SEA communicates with these partners intermittently as needed, 
and more formally in a “Critical Friends” session held at the annual state SRAS 
conference. 

The reviewers rated the state as having attained Level II on this indicator. Some 
external partnerships have been formed, and communication occurs throughout 
the year. To reach a higher status, the SEA should engage additional partnerships 
and formalize shared decision making with a clear focus on how these partner- 
ships will assist districts and schools with improvement. (Level III). 

Exemplary Evidence of High Performance 

0 Reports or documents describing procedures for creating partnerships 

0 Written documentation of active partnerships inside and outside the SEA, 
and their associated activities 

0 Data reflecting improvement activities by districts and schools in direct 
connection to these partnerships 

0 Agendas and associated materials (e.g., handouts, planning documents) 
from partnership meetings 

0 Focus group or survey data from schools or districts corroborating the 
effectiveness of services provided from partnerships, along with one or 
two specific examples 
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8 

External partners and providers 

8.2 

Providing guidance for 
tutoring and extended- 
learning time 

The SEA has written policies for assisting districts/schools (especially priority 
ones) with a variety of alternatives to help students learn. Among these proce- 
dures are documents to inform them about effective tutoring and extended-learn- 
ing time opportunities. The SEA has worked with its lengthy list of partners, 
including IElEs and regional centers, to provide this training as well as support- 
ive materials. 

The reviewers rated the state as having attained Level III because the SEA has 
gone beyond just having written information and has used its partners to train 
local personnel. The fact that this training has been made available in IEIEs is 
laudable. However, there is no evaluation plan noted to see if these ideas and 
activities are really useful. 

Exemplary Evidence of High Performance 
0 Written plans for using tutoring and extended-learning time 
0 Training materials 

0 Survey data from districts/schools noting which activities worked and why, 
using one or more specific examples 
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9 

Removal of barriers to change and innovation 

9.1 

Removing barriers to 
change 

The SEA is working with the governor and state legislature to create more 
flexibility in selected policies, particularly in alternative programs for teacher 
licensing and principal certification. Meetings are scheduled for later in the 
school year to discuss how to remove barriers such as several SEA rules that 
would provide needed flexibility for local educators. 

The reviewers rated the state as operating at Level I on this indicator. Although 
SEA is attempting to promote removal of barriers to change, a formal process 
has not yet been developed or implemented. 

Exemplary Evidence of High Performance 
0 A documented process for identifying and removing barriers to change 

0 Reports or documents describing actions taken by SEA (or partners) to 
assist districts and schools in removing barriers 

0 Data reflecting activities by schools and districts to remove barriers 

0 Survey data from districts and states corroborating the effectiveness of the 
state support, and indicating one or more specific examples 

9.2 

Creating options for 
new types of schools, 
including charter 
schools 

Two years ago, the SEA completed a formal plan for expanding options for 
charter schools and high school “career academies.” Although the academy 
proposal is still under consideration, the State Board of Education and the legis- 
lature subsequently approved more liberal criteria for student eligibility to enroll 
in charter schools and increased the maximum number of charter schools in the 
state from 75 to 150. 

The reviewers rated the state as operating at Level III on this indicator. Critical 
accomplishments included the creation of a written plan, implementation of new 
legislation, and growth in the number of new types of schools. However, any 
plan to evaluate the effects of these new options has yet to be written. 

Exemplary Evidence of High Performance 

0 A documented process for creating new types of schools 

0 Reports or documents describing actions taken by the SEA to implement 
policy changes for creating new types of schools 

0 Data describing the new types of schools created each year 

0 Test scores or other data reflecting the success of the new types of 
schools compared to traditional schools 
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9 

Removal of barriers to change and innovation 

9.3 

Expanding access to 
college level courses 
or their prerequisites, 
dual enrollment, 
courses, or other 
accelerated learning 
opportunities 

The SEA has worked with high school principals and IEIEs to encourage the use 
of Advanced Placement programs and dual enrollment programs in each high 
school. Financial incentives have been proposed for successful scoring on the 
AP tests. Public colleges now award credit based on AP exam courses, and they 
work with high schools to encourage dual enrollment programs. In addition, 
they provide training for high school teachers to ensure that the students receive 
college level courses while still in high school. 

The reviewers awarded a Level III rating even though the SEA was making 
serious efforts at implementing programs throughout the state’s high schools. 
There is not yet a plan to evaluate whether this effort is improving the number 
of students meeting performance or graduation targets. 

Exemplary Evidence of High Performance 

0 Trend lines documenting the last five years of participation by the SEA’s 
high schools in both AP and dual enrollment programs 

0 Surveys of educators discussing their successes and problems in working 
with IHEs to implement AP and dual enrollment programs, with specific 
examples 

0 Surveys of IHE’s documenting how the participation in these programs 
affects student learning and graduation rates in high schools 


10 

Incentives for change 

10.1 

Setting consequences 
for low student 
achievement and low 
graduation rates 

The SEA has completed a plan for increasing the sanctions for schools and 
districts that perform poorly for multiple years. A major component of this plan 
requires a comprehensive needs assessment conducted by a third party evalua- 
tor, and the associated identification and adoption of evidence-based programs 
in mathematics and reading. Improvement consultants and support teams have 
been assigned to assist local educators make the necessary changes. 

The reviewers rated the state as performing at Level III on this indicator. To 
achieve Level IV, the state will need to produce evidence showing that the 
added consequences have been levied and that there is some improvement. 

Exemplary Evidence of High Performance 

0 A written plan describing stages of consequences for low performance 
that continue over time based on whether the districts/schools make 
gains on their performance targets 

0 Websites or written publications communicating publicly the potential 
consequences for not meeting performance targets over time and the 
reasons for those actions 

0 School or district reports reflecting improvement efforts taken in response 
to the sanctions and the subsequent results 
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10 

Incentives for change 

10.2 

Providing positive 
incentives for 
improvement 

In enacting its plan for providing incentives to districts/schools for improved 
academic achievement, the SEA awarded, during the last school year, 45 
“Enrichment Grants” and 13 “Technology Integration Grants.” The districts/ 
schools that surpassed improvement targets were formally recognized at the 
annual Governor’s Education Conference in July. 

The reviewers rated the state as operating at Level III on this indicator based 
on the implementation of its incentives plan. To achieve Level IV, evidence 
showing that the awards have assisted the districts/schools to maintain their 
high level of performance on multiple indicators must be collected. Also to be 
collected as part of the evaluation plan, evidence that other districts/schools 
believe in the importance of the SRAS incentives. 

Exemplary Evidence of High Performance 

0 A written plan for awarding incentives to districts or schools for high 
performance 

0 Websites or written publications communicating publicly the incentives 
awarded 

0 School or district reports reflecting improvement efforts taken in response 
to the incentives 

0 Focus group or survey data from districts and schools corroborating the 
impact of the incentives in stimulating positive change, and indicating one 
or more specific examples 

10.3 

Publicly disclosing 
district and school 
performance 

The SEA has created a website that annually reports trends by individual 
schools and districts in terms of the number of achievement performance tar- 
gets met, including results by subgroup. 

The reviewers assigned a rating of Level II on this indicator. The state was 
viewed as providing only “limited data” to parents and the public. To attain 
Level III, a more informative, less passive, and less technical version should be 
available in multiple languages as needed. There would also have to be a more 
active effort to disseminate the reports to the SRAS partners and improvement 
consultants. 

Exemplary Evidence of High Performance 

0 Websites or written publications for communicating school and district 
performance publicly 

0 Data reflecting distribution to parents in English or other languages as 
appropriate. 

0 Survey data from parents corroborating the usefulness of the data 
(informative, user-friendly, timely, etc.), and indicating one or more 
specific examples 

0 Descriptions of changes made to the dissemination process based on 
feedback from partners, external partners, and school improvement 
consultants 
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11 

SRAS communications 

11.1 

Communicating with 
clear and systematic 
communication paths 
within the SRAS 

The SEA has assigned staff from its communications office the responsibility 
of developing an inter-departmental communication process. A recent internal 
evaluation indicated that SEA employees had been dissatisfied with commu- 
nications and interactions within and between departments. There was strong 
consensus that SRAS efforts were hampered as a result. 

The reviewers rated the state as operating at Level I on this indicator. At the 
present time, there is no documented plan for increasing communications, but 
it is anticipated that performance will rise to Level II when these staff complete 
the development of the communications process. 

Exemplary Evidence of High Performance 

0 A written plan for structuring, improving, and increasing communication 
within the SRAS 

0 Meeting agendas or records of inter-departmental interactions and other 
SRAS communications 

0 Reports or other documents associating communications with the 
improvement of services 

0 Survey or formative evaluation data from SEA staff corroborating 

improved communications within the SRAS, and indicating one or more 
specific examples 

11.2 

Implementing clear 
and systematic 
communication paths 
between the SEA/ 
SRAS and districts/ 
schools as well as 
significant others 

The SEA designated a liaison from its staff to write a plan on how to regularly 
transmit information regarding SRAS services, products, training programs, 
etc., to districts, regional offices, improvement consultants, and external 
partners. In return, these districts and partners in school improvement pro- 
vide information back to the SRAS for sharing. This plan contains multiple 
modes of communication including a monthly electronic newsletter that has an 
updated directory of individual contacts, school and district office information, 
and planned SRAS programs. 

The reviewers assigned the state a rating of Level III on this indicator. There 
is regular and systematic communication using the state plan. To achieve Level 
IV, evidence of improved services as a result of an evaluation will be necessary. 

Exemplary Evidence of High Performance 

0 A written plan for structuring, improving, and increasing communication 
among district staff, important external partners and the SRAS 

0 Meeting agendas and records of SRAS communication efforts 

0 Reports or other documents associating communications with the 
improvement of services 

0 Descriptions of dissemination modes used by the SRAS 

0 Survey or formative evaluation data from district staff and other partners 
corroborating improved communications by and with the SRAS, using 
one or more specific examples 
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12 

Technical assistance 

12.1 

Delivering training 
to districts and 
schools in school 
improvement planning, 
implementation, and 
monitoring 

SEA has developed a training plan to prepare schools and districts for develop- 
ing and evaluating school improvement plans (SIPs). Each identified school 
and district is required to train a leadership team once every two years. The 
SRAS has provided training materials and school improvement consultants to 
aid in this process. Accordingly, in the past two years, 100% participation has 
been achieved. 

The reviewers evaluated the state as operating at Level II on this indicator. 
Although a training process has been developed and fully implemented, it is 
limited to only one component — SIPs. By developing a more comprehensive 
process that also addresses needs assessments and implementation and evalua- 
tion of SIPs, the state would advance to Level III. 

Exemplary Evidence of High Performance 

0 A documented training plan to prepare districts/schools for improvement 
planning 

0 Training materials/tools used for improvement planning 
0 Agendas/handouts/participation lists for training sessions 

0 Survey data from districts/schools regarding use of plan, outcomes, etc., 
with one or more specific examples 

0 District/school data demonstrating systematic improvement planning 

12.2 

Providing technical 
assistance to improve 
professional practice 

The SEA has worked with its external partners, including the Principals’ 
Association, the regional centers, and the IHEs to plan the content of training 
programs regarding professional practice. The plan also includes how district 
personnel will learn about these SRAS sponsored training opportunities that 
have been planned to occur throughout the next school year. 

The reviewers believe this SEA is only at Level II even though they have 
shown great leadership in pulling together a team of institutions to provide the 
resources. The training simply has not yet been implemented. 

Exemplary Evidence of High Performance 

0 Results of a statewide needs assessment showing the kinds of technical 
assistance services relative to the improvement of professional practice 
needed and where they are needed, that is, what districts should receive 
them first 

0 Documents showing that the SRAS has carefully studied the content and 
location of training sessions so that there is a careful match with local 
personnel and their needs 

0 A list of the proposed training sessions along with a description of the 
content of the professional practices training programs 

0 An evaluation plan to see if the training programs have positively affected 
student performance and graduation rates 
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12 

Technical assistance 

12.3 

Building parent 
involvement into 
school improvement 

The SEA has identified increased parent involvement in the school improve- 
ment process as a high-priority goal. Therefore, each school is required to 
establish benchmarks for parent involvement, but strategies for achieving them 
have not been defined as part of the SRAS. 

The reviewers placed the state in Level I on this indicator. Advancement to 
higher levels will initially require a written training plan for involving parents 
(Level II) followed by implementation of the plan (Level III). 

Exemplary Evidence of High Performance 

0 A written training plan to prepare schools and districts for including 
parents in improvement planning 

0 Training materials/tools used for improvement planning 
0 Agendas/handouts/participation lists for training sessions 

0 Survey data from districts/schools regarding use of plan, outcomes, etc., 
with one or more specific examples 

0 District/school data or artifacts showing implementation of parent 
involvement plans (benchmarks, parent meeting agendas, etc.) 

12.4 

Evaluating external 
providers 

The SEA has worked with the Department of Education and several states to 
develop criteria it can use to evaluate those companies that claim they can be 
successful external providers for districts/schools. An independent group of 
state experts have been asked to review all the external providers’ proposals on 
an annual basis using these criteria. A successful provider then has his status 
approved for a period of three years until he has to submit again. Additional 
submissions must include a discussion of any work that has been carried out in 
the state during the three previous years. This process has worked for the last 
four years. 

Reviewers gave this state a Level IV rating since they had developed, written, 
and implemented a process to evaluate external providers. The SEA had even 
built in an ongoing component in its evaluation plan. 

Exemplary Evidence of High Performance 

0 List of criteria used to evaluate external providers 
0 A description of the process used to select the independent reviewers 
0 A list of the successful and unsuccessful applicants for each year 
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12 

Technical assistance 

12.5 

Implementing content 
standards that prepare 
students to take 
credit-bearing courses 
at post-secondary 
institutions and for a 
career 

The SEA is a participant in the CCSS but realized that high school staff would 
need assistance in rethinking, reorganizing, and subsequently developing new 
ways to evaluate their courses. The SEA began work with a variety of post- 
secondary institutions to ensure high school graduates would have the knowl- 
edge and skills to be successful after graduation. Regional centers provided the 
technical assistance to high school staff to ease into this new process. The SEA 
has a third party to evaluate the effectiveness of this training and the materials. 

The reviewers gave this state a Level IV rating since they had implemented 
a plan to make the CCSS alive within high schools across the state. Also, the 
SEA had an evaluation plan to ensure that they were on the right track. 

Exemplary Evidence of High Performance 

0 A report detailing the different ways high school courses could be 
restructured to make sure that students were prepared to take credit- 
bearing courses at post-secondary institutions and for a career 

0 Evaluation reports from the high school personnel who attended the 
regional center training programs in which they are asked about the 
adequacy of the training 

0 Annual evaluation reports summarizing the judgments from post 

secondary institutions about whether there is a difference in the quality 
of the preparation of high school students who are now attending their 
institutions 


13 

Dissemination of knowledge 

13.1 

Disseminating 
knowledge and/ 
or research-based 
practices 

In accord with its plan for disseminating knowledge, the SEA has established 
a process for making products and resources available to help districts and 
schools with implementing improvement activities, including manuals and 
modules on curriculum alignment and how to improve instruction. 

The reviewers rated the state as operating at Level III on this indicator, based 
on SEA’s development and implementation of a dissemination plan (which 
appears to address varied key topics for school improvement). To achieve 
Level IV, evidence that schools/districts are using the information to promote 
positive change and that the changes are improving performance targets is 
necessary. 

Exemplary Evidence of High Performance 

0 A documented process for systematically disseminating school 
improvement information to districts/schools 

0 Description of SEA activities, staffing, and resources used for 
disseminating such information 

0 Sample websites or materials (manuals, reports) made available to 
districts/schools 

0 Data or reports describing actual usage by districts/schools 

0 Survey data from district/school staff corroborating the value of the 
information provided (informative, user-friendly, timely, etc.), with one or 
more examples 
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13 

Dissemination of knowledge 

13.2 

Producing products 
and resources to help 
districts and schools 
improve 

Two years ago, the SEA received a grant from an in-state foundation to develop 
products that support school improvement. Consequently, SRAS staff devel- 
oped a website that lists available products and a “resources center” was set up 
to provide information and distribute products, such as training materials, to 
districts, schools, regional centers, improvement consultants and IHEs among 
others. The resource center routinely follows up with consumers by requesting 
an online or phone evaluation of the products used. Requests for products has 
grown significantly (by over 20%) each year. 

The reviewers evaluated the state as clearly meeting criteria for Level III for 
this indicator. Its program is documented, fully developed, and being used by 
recipients. Elowever, there is no written evaluation plan to document the effec- 
tiveness of the products in helping districts/schools meet performance targets. 

Exemplary Evidence of High Performance 

0 A listing of products made available to districts and schools to facilitate 
improvement planning (manuals, assessments, rubrics, etc.) 

0 Sample websites or other communications (manuals, reports) used to 
inform LEAs and partner groups of the products 

0 Communications (meetings, memos, etc.) with service providers to 
facilitate matching of products to school and district needs 

0 Data or reports describing actual usage by districts/schools 

0 Survey or formative evaluation data regarding the effectiveness of the 
products, and indicating one or more specific examples 


14 

Monitoring, program audits, and diagnostic site reviews 

14.1 

Conducting state 
monitoring, program 
audits, and diagnostic 
site reviews 

Recommended interventions for school improvement are based on needs 
identified during a 3-day SRAS diagnostic visit (which includes classroom 
walkthroughs, interviews with district/school staff, students, and community 
members), assessment data, financial data, and Solutions Team Findings. The 
SRAS team presents a report to the district and affected schools detailing rec- 
ommendations for changes in order for performance targets to be met. 

The reviewers rated the state as operating at Level III on this indicator. 
Although a systematic diagnostic review process is being implemented, 
remaining to be achieved is a systemic evaluation plan which would document 
that the recommendations are being followed by the districts/schools involved 
and that they are having a positive effect. 

Exemplary Evidence of High Performance 

0 A documented process for conducting site visits to identified schools and 
districts 

0 Description of SEA activities, staffing, and resources used for visiting 
schools and districts 

0 Tools, materials, and reports used in the audit process 
0 Audit reports and recommendations for individual schools or districts 
0 Reports describing results of the audits state-wide 

0 Survey data from districts’/schools’ staff corroborating the value of the 
audits for improvement processes, and indicating one or more specific 
examples 
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14 

Monitoring, program audits, and diagnostic site reviews 

14.2 

Documenting the 
status of districts/ 
schools 

The SEA annually posts on its website a listing of all schools and districts in 
the state needing substantial improvement, the number of years they have been 
identified as needing improvement, and the criteria used to identify them as 
needing improvement. 

The reviewers placed the state at Level II on this indicator. For advancement to 
higher levels of performance, the posting of district/school status would need 
to be augmented to include evidence documenting the number of districts and 
schools that have been removed from this list over time, that is, whether the 
number of districts and schools meeting performance and graduation targets are 
increasing or decreasing over time. Such a description would include informa- 
tion about the effectiveness of SRAS interventions and supports. 

Exemplary Evidence of High Performance 

0 A documented list of identified schools and districts 
0 Description of interventions used in serving the identified schools 
0 Data reflecting trends in moving districts/schools off the list over time 

0 Reports identifying interventions identified as effective for school 
improvement 

14.3 

Monitoring the 
progress of individual 
districts/schools 

The SEA has developed the Standards and Rubrics for School Improvement. 
The electronic needs assessment is completed by districts/schools not meeting 
academic and graduation standards and therefore are in some level of school 
improvement. They first complete the needs assessment and then a new school 
improvement plan that addresses the specific needs of each district/school 
based on the five critical standards from the Standards and Rubrics for School 
Improvement document: 1) School and District Leadership; 2) Curriculum/ 
Instruction/Professional Development; 3) Classroom and School Assessments; 
4) School Culture/Climate/Communication; and 5) Resource Management. The 
new plan includes financial and assessment data as well as a progress report on 
actions taken since the state’s last Diagnostic Team visit. 

The reviewers assigned the state as Level III on this indicator. The state was 
viewed as having an exemplary system that has stimulated needed reforms by 
identified districts/schools. However, there is no written evaluation plan with 
clear criteria to determine the long term effectiveness of this approach. 

Exemplary Evidence of High Performance 

0 A written plan for formatively evaluating the progress of identified 
districts/schools 

0 Description of SEA activities, staffing, and resources used for formatively 
evaluating the progress of schools and districts in meeting improvement 
targets 

0 Tools, materials, and school reports used in the evaluation process 

0 Reports or other documentation of school improvement efforts based on 
the evaluation results 

0 Survey data from district/school staff corroborating the value of the 
formative evaluation process for improvement processes, indicating one 
or more specific examples 
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Part D: Outcomes for Districts and Schools Served by the SRAS 


15 

Outcomes/results 

15.1 

Establishing student 
achievement 
performance targets 

The SEA has implanted a system of academic and graduation performance 
targets throughout the state. Analysis of state assessment data from last year 
produced the following general results: 

Of the state’s 1 12 districts, 32 (29%) did not meet their performance targets in 
both reading and math. 

Eight of the districts with the largest achievement gaps did not show progress. 

The above results along with specific information by all districts and schools 
not meeting performance standards are available on the SEA’s website. 

The reviewers placed the state at Level III on this indicator. The state met the 
criteria for a written process to implement a state system of performance targets 
for districts/schools, and it is displaying the results. However, it does not yet 
have an evaluation plan to investigate more deeply why districts are continuing 
to not meet these performance targets. 

Exemplary Evidence of High Performance 

0 A report documenting who is meeting state achievement performance 
targets by districts and schools 

0 One or more evaluation reports describing a rigorous study of whether 
the establishment of these high performance targets is reasonable and 
leading to improvement 
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15 

Outcomes/results 

15.2 

Addressing subgroup 
achievement gaps 

This year, a cross-functional team of SEA staff has forged voluntary partner- 
ships with a number of districts. The focus of the three-year partnerships is 
to close achievement gaps associated with race and income. The arrangement 
begins with a comprehensive appraisal of district improvement needs based 
on the state data system and framework for using the data in school planning. 
What emerges is a set of recommendations and a plan for improvement using 
evidence-based strategies. Each district then selects an “achievement gap 
consultant” who assists with plan implementation and professional develop- 
ment. The state department of education prequalifies these consultants and 
underwrites the cost of their service. The consultants broker services in a way 
that delivers customized resources to address the significant achievement chal- 
lenges of the district. 

The reviewers rated the state as operating at Level III, given its active assis- 
tance (via partnerships, communications, and funding achievement gap con- 
sultants) to districts and schools in identifying achievement gaps based on data 
and implementing evidence-based strategies to reduce the gaps. Elowever, there 
is no written evaluation plan detailing the criteria to be used over the next three 
years to document whether this process has been effective. 

Exemplary Evidence of Eligh Performance 

0 A written plan describing the process and requirements for schools and 
districts in using achievement gap data 

0 Description of actual services provided to identified sites by SEA staff or 
external consultants 

0 Written material (web-based or paper) identifying evidence-based 
strategies to reduce achievement gaps 

0 Agendas/descriptions of meetings or training sessions with school or 
district representatives regarding usage of achievement gap data or 
interventions 

0 Communications/records showing school or district implementation of 
interventions 

0 Test score data indicating narrowed achievement gaps 
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15 

Outcomes/results 

15.3 

Establishing 
student attendance 
performance targets 

The SEA has documented evidence to show that it can reliably and validly 
measure student attendance. It subsequently established attendance targets for 
all schools and districts. Only 62% of the targeted high schools have met their 
targets. Further analysis has revealed that these same high schools are generally 
not meeting their performance and graduation targets. 

The reviewers placed the state at Level III on this indicator. The state has not 
yet shown that it has an ongoing process for evaluating and improving the effi- 
ciency and effectiveness of its policy and procedures for connecting attendance 
rates to school improvement planning. 

Exemplary Evidence of High Performance 

0 A report documenting the meeting of state attendance performance 
targets by districts and schools 

0 One or more evaluation reports describing a study of programs designed 
to raise attendance (e.g., schedule changes, parent involvement, hiring 
additional social workers, etc.) and how that connects with school 
improvement planning 

15.4 

Establishing 
graduation rate 
performance targets 

The SEA has documented evidence to show that it can reliably and validly 
measure graduation rates for all its high schools and that it makes the results 
public on the annual report cards provided to parents and the public. 

The reviewers placed the state at Level III on this indicator. The state has not 
yet shown that it has an ongoing process for evaluating and improving the effi- 
ciency and effectiveness of its policy and procedures for connecting graduation 
rates to school improvement planning. 

Exemplary Evidence of High Performance 

0 A report documenting the number of high schools meeting their 
graduation performance targets for all students as well as subgroups. 

0 One or more evaluation reports describing a study of programs designed 
to raise graduation rates (e.g., mentoring, after-school tutoring, reduced 
class size, etc.) 
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Evaluating the Outcomes of the SRAS 

Steven Ross, C. Thomas Kerins 

Qualities of an Effective Evaluation of the SRAS 

By systematically and rigorously evaluating their SRAS, SEAs realize several important benefits. One is making 
reliable and valid judgments of the status of the services provided, that is, how fully are the services being imple- 
mented and to what degree are the expected outcomes for identified schools and districts being achieved? A 
second important benefit is using the evaluation results to document accountability by the SEA and external orga- 
nizations (e.g., providers, universities, consultants, community agencies) offering the services. A third benefit is 
demonstrating accountability to consumers, namely, the schools and districts served, students, parents, educators, 
and the general public. A fourth is developing a process for continuous program improvement by sharing evalua- 
tion feedback with all stakeholders and establishing defined improvement goals. 

To be useful, an evaluation process, first and foremost, must be valid. Validity simply means producing results 
that are reliable (e.g., accurate, consistent, replicable) and meaningful (i.e., relevant to program goals). In the case 
of the SRAS, which encompasses a broad range of services, providers, and stakeholders, validity is achieved in 
ways that strike a balance between what is practical and feasible but also sufficiently rigorous and credible. Thus, 
a highly controlled experimental study would likely provide a great deal of rigor, but would hardly be practical 
for addressing the complexities and range of SRAS services. At the other extreme, basing a judgment primarily on 
anecdotal data (e.g., “We heard at the conference that Oak County Schools are doing great this year.”) or subjec- 
tive impressions (e.g., “Our data system seems far superior to what other states have.”) may seem practical and 
feasible to some but certainly would not be rigorous and credible. 

As evidenced by the “SEA System of Recognition, Accountability, and Support Performance Management 
Rubric,” state evaluations of their SRAS focus on three main properties of the services provided: 

0 Whether or not a particular tangible product has been created (e.g., The state has a documented training plan 
and policy to prepare distinguished educators to assist districts and schools.) 

0 Whether particular actions have been taken toward achieving desired goals (e.g., The state assists districts 
and schools in implementing strategies to reduce achievement gaps.) 

0 Whether tangible outcomes have been attained (e.g., The recruitment and retention of highly effective 
administrators improved by 25% over last year). 

Directly relevant to all three attainment categories is the dimension of “quality” — to what degree or how effec- 
tively does the product, activity, or outcome meet desired standards as defined in the SRAS rubric performance 
levels? Judging quality significantly elevates the value and impact of the evaluation process. But, there is one 
“catch.” 

To judge a service, standards or criteria are needed. The SRAS rubric performance levels provide a general 
framework, but it is up to each state to identify the specific standards. What constitutes an “enhanced recruitment” 
of well qualified administrators or “greater efficiency” in resource allocation? Some rubric indicators are more 
amenable to specific (e.g., numerical) standards than others, but to have a strong SRAS evaluation, all indicators 
need further definition of what lower and higher quality products, actions, and outcomes are. 

For balancing the practicality and rigor of SRAS evaluations, a key component is having evidence to support 
judgments of quality of services. Evidence can take many forms, as illustrated in the examples provided in 
“Illustrating the Scoring System” found in the previous chapter. But, to ensure rigor and credibility, the evidence 
considered should have the following properties: 

It is tangible — information or data that can be examined, verified, replicated, and/or judged by an external 
evaluator. 
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Examples 

0 A documented plan 

0 A meeting agenda 

0 Survey responses 

0 Performance targets 

It is objective — information or data obtained from 
extant records or sources not having a personal stake 
in the evaluation results. 

Strong Examples 

0 “80% of the principals surveyed rated the sup- 
port from the school improvement teams as very 
helpful, and they were able to provide a specific 
example of how these consultants helped their 
school.” 

0 “Trend data show a 15% increase in the number 
of organizational partners that work with the 
state to provide specific assistance to schools 
identified as needing help.” 

Weak Examples 

0 “The matching of schools with support teams 
was reported to be effective.” 

0 “The Governor touted the state’s progress in 
assisting low-performing schools since he took 
office.” 

It is relevant — information or data directly connected 
to evaluation standards or goals (e.g., “SEA System of 
Recognition, Accountability, and Support Rubric”) 

Strong Example 

0 A state rates itself as being at Level IV (full level 
of implementation) in delivering training to dis- 
tricts and schools in school improvement plan- 
ning. One piece of evidence is a report by each 
district documenting the interventions identified 
as implemented in direct response to the training. 

Weak Example 

0 A state rates itself at Level IV in delivering train- 
ing to districts and schools in school improve- 
ment planning on the same indicator based on a 
synthesis of School Improvement Plans (SIPs) 
submitted electronically by schools state-wide. 
(Note that the SIPs are required of all schools 
and may not have any connection to the state’s 
achievements in SRAS training) 

It is evaluative rather than merely descriptive — infor- 
mation or data that can be judged in terms of quantity 


or quality (and, ultimately, achieving the indicator stan- 
dards). Contrast the differences illustrated on the next 
page. 

Table 1: Contrast Between Descriptive and Evaluative 
Evidence 


Indicator 

Descriptive 

Evidence 

Evaluative 

Evidence 

Building parent 
involvement 
into school 
improvement 

A copy of the 
manual used for 
training school 
leaders but no 
evidence that 
it is used by 
SEA staff or is 
available for 
LEA staff 

School reports 
documenting 
actual parent 
programs 
implemented in 
response to the 
training 

Communicating 
with clear and 
systematic 
communication 
paths within the 
SRAS 

A bar chart 
showing the 
number of hours 
of meetings 
held each month 
with service 
providers 

Survey data 
from providers 
showing how the 
communications 
were useful 
in improving 
services 

Providing a 
comprehensive 
SEA data system 

A technical 
report describ- 
ing the proper- 
ties of the data 
system 

User data showing 
increases in the 
frequency with 
which schools and 
districts access 
the system for 
improvement 
planning 


Evaluation Recommendations 

The examples in Table 1 all represent processes and 
activities that occur as part of or as a result of SRAS. 
That is, survey data may indicate positive perceptions 
by providers (e.g., universities or regional service 
centers) of the communications they received in con- 
nection with SRAS services, or a newly published 
guidebook may appear well designed and useful to 
school leaders attempting to increase parent involve- 
ment. Positive attainments in the numerous types of 
services defined by the SRAS Rubric indicators rep- 
resent important progress toward, but not necessarily 
accomplishment of, the ultimate goal of the SRAS — 
improving educational outcomes for students. Accord- 
ingly, Part D, Section 15 of the Rubric — referred to as 
the essential indicators — was created for the purpose of 
guiding evaluation of the latter outcomes. For clarity 
and convenience, we repeat the Rubric section dealing 
with student achievement in Table 2. 
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Part D-Section 15 


Evaluating the Outcomes of the SRAS 


Essential 

Indicators 

Level 1 

Level II 

Level III 

Level IV 

15.1 Establishing 
student achieve- 
ment performance 
targets 

The SEA has not 
established perfor- 
mance targets for 
districts and schools, 
nor has it established 
criteria on how to use 
the state assessment 
to identify the highest 
and lowest performing 
schools as well as the 
high progress schools 
and those schools con- 
tributing to achieve- 
ment gaps in the State. 

The SEA has estab- 
lished high perfor- 
mance targets for 
districts/schools and 
criteria on how to use 
the state assessment 
to identify the highest 
and lowest performing 
schools as well as the 
high progress schools 
and those schools 
contributing to the 
achievement gaps in 
the State. 

The SEA has imple- 
mented its high 
performance targets 
for all districts/schools 
using clear criteria to 
annually document 
their progress and to 
identify the highest 
and lowest performing 
schools as well as the 
high progress schools 
and those schools 
contributing to the 
achievement gaps in 
the State. 

The SEA has an ongo- 
ing process for evalu- 
ating and improving 
the efficiency and 
effectiveness of its 
policies and proce- 
dures relative to set- 
ting high performance 
targets for all districts 
and schools as well 
as the criteria used to 
identify the highest 
and lowest performing 
schools as well as the 
high progress schools 
and those schools 
contributing to the 
achievement gaps in 
the State. 


What makes this set of indicators essential ? The SRAS 
is ultimately about improving the achievement and 
educational success of students as well as attendance 
and graduation rates. Because these outcomes are 
“distal” or “culminating,” they may not be affected 
immediately by SRAS services. Therefore, judging 
immediate (proximal) progress solely on the achieve- 
ment of state performance targets may be misleading. 
Take, for example, a situation in which an SEA designs 
and delivers a high-quality training program to prepare 
principals for turning around low-performing schools. 
Although we would expect this program to impact 
student achievement eventually, time will be needed 
for the principals to meet and work with teachers, who, 
in turn, will need time to replace traditional strategies 
with more effective ones. So, any given SRAS service, 
no matter how well designed and delivered, may fail to 
impact essential indicators demonstrably in the short 
run. But, to be judged as successful, SRAS services as 
a whole should be producing measurable gains over 
time. 

Given these considerations, we next offer recommen- 
dations for making evaluations of states’ performance 
on the essential indicators more comprehensive and 
meaningful. 

Recommendation 1: Treat the essential indicators sec- 
tion ( Rubric Part D, Section 15) as a basic framework 


(or starting point) for evaluating state performance. 
Probing more deeply and analytically into the results 
elevates the evaluation to the next level — understand- 
ing the data and its implications. Specifically, the basic 
Rubric performance rating (see Table 2) is certainly 
relevant for accountability purposes and generating 
an overall picture of the status of the targeted dis- 
tricts/schools (i.e., the percentages achieving state 
benchmarks). But unfortunately, that information 
alone conveys little about the true effectiveness of the 
SRAS. Suppose, for example, that 100% of the identi- 
fied schools reach performance targets in a given year 
or, disappointingly, none does. Even in these extreme 
situations, the quality of the SRAS services provided 
could have ranged from poor to excellent. Factors such 
as the difficulty level of last year’s state assessment, 
changes in performance standards, characteristics of 
the schools and students, or the effects of academic 
programs implemented independently of the SRAS 
could influence educational outcomes substantially. 

So, although the overall Rubric rating is a logical 
starting point, the evaluation process should not end 
there. Recommendations 2 and 3 suggest some useful 
extensions. 

Recommendation 2: Supplement the basic Rubric 
evaluation (Recommendation 1) with follow-up analy- 
ses of probable “root causes” of successes and failures. 
Such analyses do not require expensive or intensive 
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research studies, but rather efforts to identify: (a) 
potentially successful turnaround strategies that may be 
transferred to other schools; (b) unsuccessful strate- 
gies or conditions that need replacement by alternative 
interventions; and, most fundamentally, (c) explana- 
tions of the outcome data relative to the SRAS services 
provided. Consider the following examples: 

0 School A increases its mathematics scores to 
surpass state performance targets for the first 
time in three years. Follow-up study (e.g., a brief 
site visit by SEA) reveals that the school imple- 
mented a new mathematics curriculum using 
interactive multimedia, cooperative learning, and 
after-school tutoring. 

Implication: The new program appears to have had a 
positive effect and may have potential to raise achieve- 
ment in other low-performing schools. 

0 School B increased its reading scores signifi- 
cantly over the past year. Follow-up study of stu- 
dent enrollment patterns reveals that community 
rezoning decreased the number of disadvantaged 
students who attended the school by 50%. 

Implication: The rise in test scores could be attribut- 
able, in part, to some SRAS services, but appears (for 
this year at least) directly linked to the changes in stu- 
dent enrollment. Continued evaluation is needed. 

0 School C failed to achieve state benchmarks in 
graduation rates and attendance, even though it 
received intensive SRAS services in teacher and 
principal professional development, technology 
integration, and using data for instructional deci- 
sions. Follow-up study indicates that over 60% of 
student enrollees are English language learners, 
many of whom have high absence rates due to 
returning to their home communities for extended 
periods. 

Implication: The school serves a high at-risk student 
population that may not show immediate or strong 
gains from routine SRAS services. Consideration, 
therefore, might be given to replacing or supplementing 
these services next year with more powerful interven- 
tions adapted to students’ needs. 

0 School D had several student subgroups fail to 
attain performance targets in reading. Follow-up 
interviews with the principal and literacy coaches 
reveal that the new reading and language arts 
curriculum was poorly supported by the provider 


(late arrival of materials, inadequate professional 
development, etc.). 

Implication: Although the services were directed to 
a relevant improvement need, the particular interven- 
tion adopted was ineffective and needs refinement or 
replacement. 

Recommendation 3: Supplement the basic Rubric 
evaluation (Recommendation 1) and follow-up analy- 
ses (Recommendation 2) with rigorous evaluations 
of selected interventions. Note that to achieve Level 
4 on the essential indicators (see Table 2), evidence 
from one or more program evaluations is essential. 

The overall SRAS evaluation and follow-up studies are 
limited to providing only suggestive and general results 
concerning SRAS effects. As will be discussed in detail 
in the next section, more rigorous research can pro- 
vide more valid evidence by incorporating comparison 
(control) groups, measures of implementation fidelity, 
and, in general, higher degrees of control over study 
conditions (extraneous or “confounding” factors). Brief 
illustrations of rigorous studies are: 

0 SEA releases an RFP for an evaluation of a block 
scheduling program implemented in 1 5 low-per- 
forming high schools. The RFP requests assess- 
ments of program implementation, teacher and 
student reactions, and test scores on “Gateway” 
exams compared to similar schools not using the 
program. A local university wins the award and 
conducts the evaluation research. 

0 SEA provides data and analysis support to assist 
a school district’s research department in evalu- 
ating the services of improvement consultants 
assigned to identified schools. The study involves 
documenting the types of services provided 
and specific changes in practices (corroborated 
through observations, interviews, and artifacts) 
resulting from each category of support. 

0 The SEA’s assessment department conducts a 
study of student achievement patterns in schools 
that implemented a new reading curriculum as 
part of the SRAS services. The study compares 
the reading scores of students in years prior to 
and following use of the program as it was intro- 
duced in different classes and grades. 

Characteristics of Rigorous Studies 

Accurate and Relevant Data 

Knowing that educational research is “rigorous” natu- 
rally raises expectations about its credibility and value. 
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But what does rigor imply as an attribute of a research 
study? There are several meanings. 

One meaning is that the data give a true picture of the 
phenomena being described. Simply put, the results 
will be complete, trustworthy, and directly address the 
research questions of interest. Suppose for example, 
that the question of interest is whether, after receiv- 
ing professional development made available through 
SRAS, teachers use more student-centered instruction. 

Weak Evidence 

0 Teachers liked the professional development 
activities 

0 The providers of the professional development 
believed the offerings to be successful 

0 SEA staff observed the professional development 
offerings and rated them positively 

Suggestive Evidence 

0 Teachers express more favorable attitudes toward 
student-centered instruction 

0 Teachers indicate that they use more student- 
centered instruction than in the past 

0 Principals and grade-level leaders indicate 
observing more frequent student-centered 
instruction than in the past 

Strong Evidence 

0 Systematic observation by independent observers 
shows significant increases in student-centered 
instruction relative to the baseline (prior to the 
professional development) and over time. In 
teacher interviews, there is strong consensus that 
the professional development was effective. 

Internally and Externally Valid Experiments 

Experimental research differs from other types of stud- 
ies by focusing on causal interpretations of program or 
“treatment” effects. A key component of experimental 
design is comparing treatment and control groups, with 
group assignments, preferably, made randomly. 

Internal validity. One quality of rigorous experiments 
is possessing high internal validity. This means that 
the outcomes measured (e.g., teaching behaviors, test 
scores, graduation rates, etc.) can be confidently attrib- 
uted to the intervention and not to extraneous factors. 

Higher Internal Validity Example 

SRAS services are helping low-performing schools to 
implement a new after-school tutoring program called 


“Afternoon T.” In the interests of supporting a rigorous 
research study, the participating schools agree to make 
the program available through random assignment to 
half of the students who sign up. The other half are 
assigned to participate in the traditional after-school 
program. At the end of the year, test score gains are 
found to be significantly higher for the Afternoon T 
group than for the traditional group. Because all condi- 
tions were exactly the same for the two groups except 
for the program received, it is concluded that After- 
noon T was effective in raising achievement. 

Lower Internal Validity Example 

Students who volunteer to participate in a new after- 
school tutoring program, “Afternoon T” are compared 
to those who don’t sign up. At the end of the year, test 
score gains are found to be significantly higher for the 
Afternoon T group than for the control group. While 
these results are encouraging, they could be due to 
motivation and parental interest being higher for the 
volunteer group. Also, the volunteer group received 
extra tutoring whereas the control group received 
none. The effectiveness of Afternoon T as an after- 
school program, therefore, is only suggested. 

External validity. A second dimension of experi- 
mental rigor is high external validity. This means that 
the findings are generalizable to the types of situa- 
tions (e.g., schools, students, teachers) of interest. 

For example, suppose that a wealthy pharmaceutical 
firm awards a large grant to each elementary school 
in a small city to hire three full-time social workers, 
a school psychologist, and a school nurse. An experi- 
mental study shows that, over the next two years, 
student attendance is significantly higher at these 
schools than at matched comparison schools. Although 
it seems likely that the added personnel were effec- 
tive (i.e., the cause of fewer student absences), the 
special circumstances and excessive costs involved in 
this “intervention” reduces generalizability to other 
schools. Thus, external validity would be low com- 
pared to a more affordable program. 

Evaluating Service Quality Through District/ 
school Case Studies 

Regardless of what an overall state-wide evaluation of 
the SRAS might indicate, additional insights of impor- 
tance can be obtained at the individual school and 
district levels. In this section, we suggest strategies for 
conducting simple “case studies” of the SRAS services 
provided for districts/schools in relation to changes in 
educational outcomes. 
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Creating a Profile of Services 

Which SRAS services were provided to individual 
schools? What were the intensity, duration, and quality 
of each service? Did any of the services appear to be 
effective in producing desired changes? Answering 
these questions is not only relevant to the evaluation of 
SRAS in general, but most importantly, reveals what 
is being done to what effect in particular contexts. 

For example, it may be found that for some schools, 
“technical assistance” in recruiting skilled teachers pri- 
marily consists of weekly phone calls or emails from 
an SEA staff member. For other schools, the support 
may be two-day site visits from marketing consultants 
and union representatives who map out a systematic 
recruitment plan and benchmarks. We would expect 
the latter, much more intensive type of service to be 
more effective in achieving goals, but is that actually 
the case? Knowing the answer will inform decisions 
on whether to continue that service another year or try 
something different. 

The first step in conducting a case-study profile is to 
document the services provided to the individual site 
(school or district). We suggest a simple checklist 
(preferably, an online version), that lists each school 
and district receiving SRAS services, followed by 
specification of: 

1. Nature of service (brief description of what 
was done by whom, when, and where) 

2. Focus of service (e.g., curriculum alignment, 
using assessment data, educating English lan- 
guage learners, etc.) 

3. Intensity of service (e.g., onsite, workshop, 
email, etc.; staff and other resources used ) 

4. Duration of service (number of days, one- 
time vs continuous) 

5. Effectiveness rating: Was the service associ- 
ated with positive educational outcomes? (Yes, 
Probably, No) 

6. Explanation of effects (If effectiveness 
rating was “Yes” or “Probably,” describe the 
evidence.) 

Although such checklists will not prove definitely the 
effectiveness or ineffectiveness of particular services, 
they should provide valuable and readable information 
for (a) profiling each district/school with regard to the 
services provided, and (b) relating each set of services 
to educational outcomes. To enable a stronger analysis 


of services’ effects, an extension of this process is sug- 
gested below. 

Relating Service Inputs to Outcomes 

Simply because a particular service encompasses many 
communication contacts or hours doesn’t mean that it 
will be more effective than one with more limited time 
or activity. Realistically, effectiveness will be influ- 
enced by multiple factors interacting in combination 
with one another. To help identify what input levels 
are associated with desired results, we suggest supple- 
menting the basic case study framework just described 
with a rubric evaluation of three dimensions of each 
service listed: Intensity, Duration, and Quality. If, for 
example, the ratings showed that a particular service 
(e.g., technical assistance for data usage) was limited 
to a one-hour workshop, only modest changes (if any) 
might be expected. On the other hand, if a different 
service (assistance in establishing partnerships) was 
consuming extensive time and resources but produc- 
ing little change, then its continuation would likely be 
re-evaluated. Some ideas for such a rubric are offered 
in Table 3. We suggest that refinements be made by 
each state based on preferences and experiences from 
application trials. 
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Table 3. Sample Rubric for Evaluating Service Intensity, Duration, and Quality in Case-Study Profiles 


Service Activity 
Dimension 

Not 

Rated 

Low 

Moderate 

High 

Intensity 

-Not applicable due to 
lack of data 

-Minimal resources 
provided (materials, 
personnel) 

-Few staff or experts 
(1 or 2) involved 
directly 

-Mostly distance 
communications 
(emails, phone calls, 
publications) 

-Small scope (affect- 
ing only a few 
educators, students, or 
parents) 

-Potential indirect 
links to outcomes 
(e.g., impacts knowl- 
edge of target group) 

-Moderate resources 
provided (material, 
personnel) 

-Several staff or 
experts involved 
directly 

-Some face-to-face 
interactions (site visits, 
meetings, consults), 
possibly combined 
with off-site communi- 
cations (calls, emails, 
publications) 

-Moderate scope 
(medium-scale 
intervention affecting 
multiple educators, 
students, or parents 

-Potential indirect or 
direct links to out- 
comes (e.g., impacts 
attitudes or behaviors 
of target group 

-Extensive resources 
provided (materials, 
personnel) 

-Many staff or experts 
involved directly 

-Multiple face-to-face 
interactions (site visits, 
meetings, consults, 
etc.) 

-Comprehensive scope 
(large-scale interven- 
tion affecting many 
educators, students, or 
parents) 

-Potential direct link 
to outcomes (e.g., 
impacts behavior of 
target group) 

Duration 

-Not applicable due to 
lack of data 

-Limited time (< 3 
days total) 

-Restricted time 
(extends less than 3 
months) 

-Moderate time (> 3 
days total) 

-Somewhat distributed 
time (extends over 3 
months) 

-Extensive time (>10 
days total) 

-Distributed time 
(extends over half of 
the year) 

Quality 

-Not applicable due to 
lack of data 

-Weak implementation 

-Not favorably 
received by district/ 
school (supported by 
evidence) 

-Adequate implemen- 
tation (supported by 
evidence) 

-Mostly favorably 
received by district/ 
school (supported by 
evidence) 

-Strong implementa- 
tion (supported by 
evidence) 

-Very favorably 
received by district/ 
school (supported by 
evidence) 
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Working with External Evaluators 

Whether a state chooses to work with an external eval- 
uator will depend on several factors. One obviously 
is budget, as an external evaluator is likely to be more 
costly than performing the work in-house. However, 
if work can’t be completed effectively by SEA staff 
due to lack of time, resources, or expertise the ultimate 
cost could be many times greater than if an external 
evaluator were used. So, a second factor is prepared- 
ness and readiness for internal staff to complete the 
tasks required. A third factor concerns the nature of the 
evaluation study being conducted. 

In general, completing the basic rubric (“Study I”) and 
collecting follow-up data relating to the essential indi- 
cators (“Study II”) are activities that states typically 
should be able to perform internally and benefit from 
by being engaged in analyzing and evaluating their 
own SRAS efforts. An independent evaluator might be 
employed (with only modest cost incurred) to review 
the evidence and corroborate the state’s self-ratings. 
Conducting rigorous studies of specific interventions 
(“Study III”) clearly is the most appropriate context for 
engaging external evaluators. One advantage of having 
external evaluators is that they have needed expertise 
in working with data, research design, and statistical 
analysis. A second advantage is that school and district 
staffs are likely to be more candid with external evalu- 
ators than with state department personnel. A third 
advantage is the independence of external providers 
who have no personal stake in the effectiveness of the 
services being evaluated. 

Working with external evaluators should neither be 
very costly nor difficult if a number of simple steps are 
taken: 

• Use a systematic process to select the evaluator. 
Whether or not a formal “Request for Proposals” 
(RFP) process is employed, it is important to 
review the candidate’s prior work and references 
from former clients. 

• Establish a clear plan of work and budget. 
Important to the plan is specifying the research 
questions that need to be answered. Such ques- 
tions take the form of: “What are the effects of 
the literacy program in raising student achieve- 
ment?” “Do new teachers who received in-ser- 
vice mentoring stay in the field longer than those 
who did not receive mentoring?” Ensure that the 
study addresses these very questions (see next 
step). 


• Monitor the study by meeting with the evalua- 
tors regularly, reviewing progress reports, etc. 
Obviously, once a study is completed, it is too 
late to refine it. 

• Work with the evaluator to disseminate the 
results to different audiences. The value of good 
research is that it informs both policy-makers 
and practitioners about which services and pro- 
grams work and which do not. 

Concluding Comments 

As occurs with virtually all endeavors in education, 
the benefits of evaluation directly relate to the effort 
and resources put into it. Making snap, subjective 
judgments to get the evaluation requirement “over 
and done with” or to make something mediocre look 
good (to avoid ruffling feathers) ultimately benefits 
no one — least of all, low-performing schools — and 
actually can be harmful. If a service isn’t function- 
ing up to expectations, a positive evaluation errone- 
ously communicates that all is going well and should 
be left alone. A truthful negative evaluation, on the 
other hand, provides the direct stimulus for making 
that service better over time. And, when a service 
is working well, credible evaluation results serve to 
recognize and sustain it. SRAS evaluations represent 
a type of service. The key to their working well, as we 
have emphasized, is using relevant evidence to make 
informed judgments relative to goals or standards. 
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